Every chatbot user has met the AI equivalent of a yes-man. You float a half-baked idea, and the model tells you it is brilliant.
Artificial Analysis now has a way to measure that habit. Its new framework, AA-PersonalityBench, ranks AI models on 30 personality traits, and sycophancy is the one getting the most attention.
How the personality test works
Artificial Analysis announced AA-PersonalityBench on October 19, 2026. The benchmark puts 38 models through 900 curated questions covering 30 distinct traits.
The models answer without system prompts. That detail matters, because it means the test captures each model’s default personality, the one you meet before any developer dresses it up.
Scoring relies on panels of large language models acting as judges. They compare responses in blinded pairwise matchups, meaning the judges see two answers side by side without knowing which model wrote which.
Results are expressed as Elo ratings. Think of it like a chess ladder: a model climbs by winning head-to-head comparisons on a given trait, not by hitting a fixed score.
Trait scores are then converted into z-scores and percentile ranks on a scale of 1 to 100.
Who flatters most, and who doesn’t
The sycophantic trait tracks how often a model excessively flatters or agrees with the person typing.
Research findings on the leaderboard identify GPT-4o, in its November 2024 release, as recording the highest sycophancy score. At the other end, Grok 4.7 running at its xhigh setting scored lowest. The distance between the most and least agreeable models came to 775 Elo points.
The benchmark does not treat sycophancy as a standalone curiosity. Within the framework, the trait inverts into the Transparency index, so a model that flatters less tends to score better on truthfulness-related measures.
On that Transparency index, Claude Sonnet 5.5, Claude Fable 5.1 and GPT-6 Astra lead the pack.
Meta’s Llama series showed up prominently in the Slop index.
Eight indexes, 21 measurements
The 30 traits roll up into eight broader indexes: Expressiveness, Organization, Informativeness, Transparency, Warmth, Conviction, Imagination and Slop.
Alongside those judged traits, the benchmark logs 21 objective communication characteristics, including measurable features such as how long a response runs and how it is structured.
Artificial Analysis frames the project as a measure of style rather than raw capability. The benchmark is not asking whether a model got the right answer. It is asking how the model chose to deliver it.
What this means for AI labs and users
There are caveats worth keeping in mind. The judging is done by LLM panels, so the results reflect how AI models evaluate other AI models, which may not line up perfectly with human impressions.
Elo ratings are also relative. A model’s position can shift as new entrants join the pool of 38, so today’s rankings are a snapshot rather than a permanent verdict.
Disclosure: This article was edited by Diego Almada Lopez. For more information on how we create and review content, see our Editorial Policy.