Cohort: simulated and real students for testing tutors

A new tutor should not learn its job on the students who count on it. First it should meet many students who know different things and make different mistakes. Then it should show, with real people, that it helps them learn.

Cohort is being built at United Education Labs to give every tutor both tests: simulated students at scale, and panels of real students from around the world.

Two kinds of students

• Simulated students. AI agents that act like real learners, each with real knowledge and real mistakes, so a tutor can face many students before it faces one person.
• Real student panels. Students from around the world, of every age, background and language, who choose to take part. They form the test and comparison groups that show whether a tutor really helps people learn.

What it will do

• Test at scale. Every new tutor meets many simulated students first.
• Check against real learners. What the simulations find is meant to be checked with real students, so the simulations stay true to life.
• Look for learning. The question is whether the student learned, not whether the tutor gave the right answer.
• Protect real learners. Weak teaching is meant to surface in simulation, so the students in a panel only meet tutors that have already been checked.

The idea behind it

Learning is the measure: what a person can do after, that they could not do before. Simulated students make testing fast and wide. Real students keep it honest. Cohort is meant to bring the two together.

How it fits

Cohort is one of three Labs tools that tell us, and everyone else, whether a product helps people learn. Delta is designed to use Cohort to measure how well AI tutors teach, by how much students learn. Proof will test every product against the research on learning. Read more on the Labs page.

Where it stands

Cohort is in development inside United Education Labs. We will release it when the work is ready, and you will see it on the release ledger when its turn comes.