There were four parts to Ashoka's Challenge Review Process - Community Review, Shortlisting, Lifeline Review, and Judging. Community Review took place earlier in the Challenge timeline and helped us determine our 122 Challenge Pioneers. Lifeline Review, Shortlisting, and Judging played out during the last stage with Pioneers full entries in hand. Our reviewing culminated in the announcement of ten Champions of Children's Wellbeing on October 12.
The first stage of evaluation relied heavily on Community Review – a peer review process powered largely by entrants themselves. Through this process, we moved from 286 Challenge entries to 122 Pioneers. Overall, we saw 130+ people engaging with other entries to the Challenge - offering 1400+ reviews and 800+ comments. 21 of the 130+ community reviewers were “Trusted Reviewers” - folks that a.) read and reviewed at least 25 entries, and b.) had a track-record of positive engagement of their peers. As we worked to determine which entries were “in” and which were “out”, Trusted Reviewer scores (in aggregate) played a very important role.
The upside of this approach is that it built cohesion across the network and created opportunities for individuals to showcase network leadership. The risk is that community perceptions of quality might not mesh exactly with those of Ashoka. Given the intention to cultivate network ownership of the emerging framework, we thought it was worth the risk.
We assembled an internal “Shortlisting” team of Ashokans core to this broader engagement. This shortlist read and discussed the “Top 60” entries from the challenge. The intention was three-fold:
- To offer a bit of extra attention and narrative feedback to those entries identified by peers as particularly compelling and relevant - understanding that the selection of this “Top 30” is tentative and subject to revision.
- To assist colleagues as they plot the evolution of the framework and the network respectively.
- To narrow the field of entrants to the ~20 Finalists we'll pass on to official Challenge Judges.
The secondary entry period closed on August 17 and we entered a period of “Lifeline Review” - a process of re-evaluating the 92 Pioneers that *weren’t* in the top 30 following “Community Review.” We divided responsibility for the 92 reviews across six groups – each group comprised of an Ashoka Staffer, a Trusted Reviewer identified during Community Review, and a Network Partner - an established institution that helped initially source Challenge participants. Reviewers scored the entries individually and then discussed their aggregate scores during hour-long calls. From each subset of ~15 entries, groups identified five entries that they passed onto the shortlisting team. Five entries from each of the six groups yielded the “Next 30” that were considered by the Shortlisting Team.
In the final month, Ashoka and RWJF worked together to determine a panel of 5-9 judges. The judges were already be familiar with Children’s Wellbeing Framework - thanks to their roles as “summer session” participants that helped us validate the framework, social media “influencers”, sector researchers, or Network Partners. The judges were people who combined a depth of expertise, a distinct perspective, and an eagerness to participate in the October Build Event in Chicago.
The Judges individually reviewed each of the 20 Finalist entries. We concluded by convening in a collective meeting to talk through what they read, consider the framework, and select our 10 Champions of Children's Wellbeing.
The judges selected 10 Champion of Children's Wellbeing. See who was selected here.