This article was originally published in The Texas Tribune.
The number of requests to rescore students’ STAAR tests nearly tripled from last year as more Texas schools sought reviews of written responses on the reading exams, thousands of which were initially graded by computers.
The Texas Education Agency began using an automated scoring engine in December 2023 to grade open-ended questions on the State of Texas Assessments of Academic Readiness, including on the reading and English tests, with humans manually reviewing a quarter of exam scores.
In spring 2024, the state had 2,758 requests to regrade open-ended response scores on reading exams. Then, 21,620 requests last year. This spring, that jumped to 64,489.
Meanwhile, the number of corrections doubled compared to the previous year. The state improved more than 13,300 reading scores after the reviews. Of those, humans initially assigned grades to 42% while computers graded 58%. Humans conducted all re-scores.
Allison Matney, director of the Texas Center for School Accountability, said historically, it was rare for district administrators to request STAAR rescoring. She attributed the increase to greater awareness about the automated scoring and review process.
“That traction of superintendents talking to each other — (and) media continuing to talk about it — made more districts aware that not only can they do this process, but actually they should, and they need to,” Matney said.
The state’s use of automated scoring for open-ended responses was initially criticized after a spike in students earning zeros on writing prompts in the first round of exams graded by computers. Agency officials, however, pointed to other causes for the low scores in 2023, including that many students were re-taking the exam.
State officials have stressed that automated computer scoring is different from generative artificial intelligence, such as ChatGPT or Google’s Gemini, as the program does not “learn” from one response to the next.
Agency spokesperson Jake Kobersky said scoring essays by hand would require four to five times the number of human scorers and cost an additional $15 million to $20 million per year. It would also result in delayed results for districts, he said.
“TEA uses a hybrid model where human experts drive the scoring. … The result is a system that keeps human judgment at the center of scoring, applies it where it matters most, and delivers results to districts and families faster than hand-scoring every response would allow,” Kobersky said.
More than 3.2 million Texas students took the reading STAAR this spring, which includes third through eighth graders, as well as those who took the high school level English I and English II end-of-course assessments. In total, less than .5% of total open-ended responses on reading exams saw changes.
TEA officials began incorporating automated scoring after the state redesigned the STAAR to include fewer multiple-choice questions and more open-ended questions — which are also known as constructed responses.
To develop the computer scoring, the TEA gathered thousands of responses that were previously scored by people. From the sample, the computer learned the characteristics of responses, and it was programmed to assign the same scores as a human.
More than 250 public charter networks and school districts, including Cypress-Fairbanks, Dallas, Katy and Northside, submitted rescoring requests this year. Nearly 90% had at least one submitted reading exam see a score increase, according to agency data.
Rachel White, an associate educational leadership and policy professor at the University of Texas at Austin, said it’s frustrating and sad that school administrators must spend time and money analyzing their students’ results and submitting thousands of rescoring requests.
“The tripling of requests to me says educators who read their students’ essays are increasingly disagreeing with the scores that are assigned,” White said. “And so whether or not that score from TEA is a machine or a person, that to me just indicates that there is a bit of a trust problem in how these are being scored.”
Public campuses and districts receive A-F accountability ratings based on the number of students who meet certain performance levels on STAAR. When campuses underperform on the test for multiple years, state law allows Texas education commissioner Mike Morath to take action, including by installing a conservator or replacing a district’s locally elected school board.
“If an instructional leader determines that your test was scored a certain value, but they really believe it should have been a couple of points higher, it could bump you up from failing to passing,” Matney said. “For an individual student, that could mean anything from now being able to graduate to now (they) don’t have to do remediation hours in the next grade level.”
TEA automatically rescored extended constructed student responses at no cost if the exams were one point from qualifying the next performance level. Those four levels, which reflect how students tested against their grade level, are: did not meet, approaches, meets and masters.
Officials from some districts said the TEA’s rescoring process for students on the cusp of meeting the next performance level freed up their budgets and allowed them to submit more students for re-scoring. One said, since TEA automatically rescored students who were one point away from the next level, they submitted students who were two points away.
In a statement, Dallas ISD officials said the district submitted a rescoring request because they felt students benefit when human scorers review their answers. In total, the district submitted more than 5,100 reading tests for rescoring, and 34% returned with a higher score.
Dallas ISD officials estimated that the combined automatic and district-requested re-scores helped five campuses in the state’s second-largest school district improve their overall ratings before the state’s public release. One campus moved from a D to a C rating and four campuses moved from C to B, according to the DISD statement.
School districts must pay $50 per test if they request rescoring and the student’s score does not change. The rescore is free to districts if a score changes.
North East ISD spokesperson Aubrey Chancellor said the district did not submit any requests after examining multiple students’ scores and determining doing so would not be beneficial to scores or campus ratings.
The Texas Education Agency’s rescoring process occurred before the release of A-F accountability ratings in August.
Disclosure: Google and The University of Texas at Austin have been major financial supporters of The Texas Tribune. The Texas Tribune is a nonprofit, nonpartisan news organization that is funded in part by donations from members, foundations and corporate sponsors. Financial supporters play no role in our journalism. Find a complete list of them and more information about our financial and disclosure policies here.
This article first appeared on The Texas Tribune.