By Isabel Andrade Moreano, Principal Product Manager, Khan Academy Kids
Khan Academy Kids is a beloved early learning app, reaching over 2 million children monthly in homes and schools. Built by the team behind Duck Duck Moose, Khan Academy Kids draws on over a decade of expertise creating delightful learning experiences for young children.
In 2022, we embarked on an ambitious new project in collaboration with the Gates Foundation and MDRC. As part of the Measures for Early Success Initiative, we’re reimagining pre-k assessments by making them more playful, nimble, effective and actionable.
Designing assessments presents unique challenges compared to learning experiences. While learning activities can be flexible and exploratory, assessments must prioritize validity and reliability, significantly constraining creative freedom in design. In this article, we reveal some of the principles we’ve distilled for creating assessments that both measure and delight.
Minimizing Distractions Without Losing Our Whimsy
The Khan Academy Kids world is distinctly whimsical – with bold colors, slightly crooked lines, and a blend of the familiar and fantastical. In designing assessments, we’ve faced a constant tension: how do we minimize distractions while preserving the wonder that makes our world special? Two examples illustrate this delicate balance:
Backgrounds
Our signature backgrounds presented an interesting challenge. For some tasks, such as comparing heights, uneven backgrounds interfered with students’ ability to make accurate height comparisons. In other tasks, such as identifying which shelf has more or fewer objects, the slightly crooked shelves had no impact on performance.
Which friend is taller? Tap them.
The stairs in the background interfered with height comparisons, leading us to simplify the background scene.
Which shelf has more? Which shelf has less?
The slightly crooked shelves had no impact on performance.
Audio
Audio presented another design challenge. In one of our pattern-making activities, we initially gave each monster a unique sound effect – from playful “huba hubas” to enthusiastic “rawrs!” Because they were delighted by these sounds, children spent more time tapping monsters to hear noises than focusing on recreating patterns. When we removed the sound effects but kept the monster characters, assessment scores stayed consistent – revealing we could maintain visual whimsy without auditory distraction. Play-testing with children in a wide range of centers was helpful for us to determine whether a given design element (asset, background, or sound) helped or hindered students’ performance on an assessment task.
Look at the pattern and copy it below.
The monster’s unique sound effects interfered with the assessment task, but the playful visuals did not.
Designing for Intentional Responses
Assessment and learning activities differ fundamentally in how they handle guessing behaviors. In learning activities, exploration and mistakes are natural parts of the discovery process, supported by immediate feedback. In assessments, however, guessing introduces noise into the data, especially since children can guess repeatedly without consequence given the absence of feedback. We’ve been exploring the following two systems for minimizing unnecessary noise in assessment data:
Response Methods Matter
Preliminary analysis revealed an unexpected pattern: activities involving drag-and-drop interactions correlate more strongly with other assessment items than those requiring simple taps. We hypothesize this stems from the intentionality of dragging – children tend to tap randomly but seem to approach drag-and-drop tasks with more deliberation.
Tap only two bugs. Drag only two toys to the water.
Preliminary data suggests dragging tasks show higher inter-item correlations compared to tapping tasks.
Rethinking Answer Choices
Increasing the number of answer choices reduces successful guessing but introduces new challenges. For young children especially, multiple choice questions tax working memory and risk conflating content knowledge with executive function. Instead, we’re exploring designs where choice complexity feels invisible. For example, line-matching activities (“match all the shapes”) naturally incorporate more options than simple selection tasks (“find the matching shape”), while maintaining a clear, focused interface.
Drag the matching shape to the basket. Match all the shapes.
Line matching naturally incorporates more options than simple selection tasks, while maintaining a clear interface.
Through careful piloting and kid-testing, we’re discovering that playful assessment design isn’t about choosing between engagement and measurement – it’s about finding creative solutions that serve both. As we continue this work, we remain committed to collaborating with educators and families to create assessments that both illuminate children’s progress and preserve the joy that makes Khan Academy Kids special.
If you’re interested in piloting an early version of Khan Academy Kids’ new pre-K assessments, please contact [email protected].
