Over the past few weeks, the nation saw something that doesn’t happen very often: Rival tech leaders agreed that artificial intelligence development needs to slow down. They are talking about AI’s potential to harm humankind, but education technology developers need to think this way about AI-based ed tech, too.
AI wields enormous influence over K-12 education, and it’s growing faster than educators, policymakers and the ed tech industry can check it. About 60% of teachers use AI tools to help prepare for class, create worksheets or tailor materials for disparate students’ needs, saving previous time. With the right guardrails in place, educators can be sure these lessons are accurate and grounded in research-based best practices for K-12 teaching. But without them, students are vulnerable.
Get stories like this delivered straight to your inbox. Sign up for The 74 Newsletter
Consider a teaching practice as fundamental as choosing a paragraph or a book for a student. Over 40 years ago, MetaMetrics took this seemingly simple (but actually quite complex) task head-on and developed the Lexile Framework for Reading, which places reading ability and text complexity on the same scale. Over 35 million students in all 50 states receive a number followed by an L on their standardized test score report, representing their Lexile measure, or ability to read and comprehend text. In addition, over 100 million books, articles and pieces of text have been officially measured for complexity.
Connecting students and reading materials with a common measure lets educators understand where students are, choose appropriately challenging materials and track growth.
Today, teachers are increasingly using large language models like ChatGPT, to adapt text for different reading levels in their class. They paste a passage into a mainstream AI model, ask for its Lexile measure and receive a remarkably confident answer. There’s just one problem: It’s usually wrong. And “in the ballpark” isn’t good enough either.
Large language models are skilled at generating language because they predict what sounds right based on patterns learned from enormous amounts of text. But they weren’t built to estimate complexity or reading difficulty. When a tool generates a Lexile-like number, it isn’t basing it on the research and billions of officially measured words behind an actual Lexile score. It’s guessing the next most likely word or number based on patterns it was trained on. Sometimes that guess is close, and sometimes it’s way out in left field, with no obvious way for an educator to tell the difference.
MetaMetrics recently published research describing the results of over 20,000 text analyses, testing ChatGPT, Gemini, Claude and Copilot on their ability to accurately measure the difficulty of 1,807 passages. The results were haphazard and inconsistent. For example, the four AI tools ranked one passage as having a level of difficulty spanning three to four grade levels. When tested again, the same tools produced an entirely different Lexile measure for the same passage. Asking any of them to then make that text more or less complex for readers of different abilities only risks multiplying the problem.
Overall, the research found that only 3% of the Lexile measures estimated by the four AI tools matched the official Lexile measures; in more than 60% of cases, the AI-estimated Lexile measures fell outside ranges optimal for student learning; and the four tools differed substantially from one another.
Comparing a student’s Lexile measure against that of a text, validated through decades of empirical research, gives educators a meaningful starting point for choosing reading material, introducing productive challenge and measuring progress. A confident guess, even one that sounds completely plausible, can’t do that.
To help address this, experts behind the Lexile framework are exploring how to integrate validated text measurement with the AI-powered tools educators increasingly use, including piloting a ChatGPT app that conducts valid Lexile text analysis rather than guessing. The goal is to help educators adapt materials efficiently while ensuring that what they put in front of students is appropriately challenging.
That matters because, one way educators and ed tech developers help students become better readers is by challenging them with increasingly complex text. Recent NAEP results make that particularly urgent for middle schoolers. Students need to progress toward whole books with longer and more complex sentences, new vocabulary and more complicated comprehension demands. As students encounter increasingly challenging texts on one side of the scale, their reading ability should grow on the other. That’s the power of measuring students and content together.
AI leaders are right to want to slow down enough to add guardrails. Ed tech developers of AI-based learning materials need to do the same, and educators using these tools should demand it. Ed tech providers and learning platform developers should keep pushing AI forward while being clear about where it needs an underlying research foundation. This includes training their models with official scales to ensure accuracy.
When ed tech developers combine the power of AI with learning science grounded in research, schools and districts can quickly deploy tools and teaching practices proven to improve learning outcomes. And that’s a home run for students.