The AI Model That Promised to Crush GPT-6 Has a Surprising Weakness

We test Inception's Mercury 3 against the fastest autoregressive models from Anthropic, Google Gemini, OpenAI, and Cerebras to find out

---

On October 19, 2026, Inception Labs launched Mercury 3 with a bold pitch: a diffusion language model that beats GPT-6 Luna and Claude Haiku 5.5 on benchmarks while running 4 to 8 times faster, at more than 1,000 tokens per second (Inception blog, "Diffusion at Scale: Introducing Mercury 3", Oct 19, 2026).

Faster and smarter than the big labs' speed models. That is the kind of sentence that makes a benchmark nerd reach for a credit card. So I did. About $34 of API spend and more than 48,000 model calls later, here is what I found.

Short version: Mercury 3 is genuinely fast and genuinely good. It is not the fastest thing on the menu, it is not the cheapest per right answer, and it has one very odd habit.

Diffusion LLMs in one paragraph

Normal (autoregressive) LLMs write like a typist: one token at a time, left to right. A diffusion LLM writes like a sculptor: it starts with a rough block of placeholder text and refines many tokens in parallel over a few passes, until the answer "comes into focus." In theory that means far fewer sequential steps, and so much more speed. If you want the longer explainer, I wrote one back when Mercury first launched: What is Diffusion LLM and why it matters (Feb 2025).

Here is the 2026 version doing its thing, on the same kind of prompt I used in 2025 ("write a Python function that splits an image into left and right halves"):

Across 30 such captures, answers resolved in 5–11 visible steps, in well under a second at the median. One detail the marketing animations skip: refinement is not a smooth fade-in. In every one of the 30 calls, at least one step moved further from the final text before coming back. The sculptor occasionally un-chisels.

The two tests, in plain English

I picked two jobs where speed actually matters to a person waiting on a screen.

Test 1: "Write me this function" (code generation). Each model gets a programming task, for example "write a Python function that checks whether any two numbers in a list are closer to each other than a given threshold." We run its code against hidden tests. Pass or fail, no partial credit. That's 637 problems from three well-known sets (HumanEval+, MBPP+, and LiveCodeBench), 5 tries each, for every model. This is the coding-assistant use case: you ask, you wait, you hope.

Test 2: "Answer this question from our database" (a SQL agent). The model gets a real database and a question like "What is the total amount of money spent on food?" (from BIRD, a standard text-to-SQL benchmark). It can't see the data upfront. It has to look around: list tables, peek at columns, run a few test queries, then submit the query that answers the question. Up to 8 turns. We check whether its query returns the same rows as the official answer. 150 questions, 3 tries each.

Why an agent? Because agents are where speed compounds. If one call takes a second and the agent needs six of them, you're waiting six seconds. A model that is fast per call but needs more calls can lose the race. More on that in a moment.

The contenders, each the "fast" tier of its maker:

• Mercury 3 (Inception, diffusion), at four reasoning settings from instant to high
• Claude Haiku 5.5 (Anthropic), GPT-6 Luna (OpenAI), Gemini 3.5 Flash-Lite (Google)
• gpt-oss-120b on Cerebras: an ordinary autoregressive open model running on very fast specialised chips. This one is my control. If Mercury is fast because of diffusion, it should beat a normal model on fast hardware. If it's fast because of serving, it won't.

Same prompts, same settings, randomised order, everything measured end-to-end over the public APIs from one machine. Full code, configs and raw results: github.com/zbruceli/mercury3.

Finding 1: Speed is real, but it is not unique

On code generation, Mercury 3 at its best setting (low) passes 86.0% of problems with a median response time of 0.83 seconds. That ties Claude Haiku with thinking off (86.1% at 0.97 s) and edges out GPT-6 Luna with reasoning off (83.2% at 1.28 s).

Then the control walks in. gpt-oss-120b on Cerebras passes 84.9% at 0.34 seconds. Within about a point of Mercury, at well under half the wait.

Raw generation speed tells the same story. Measured as visible text streamed after the first token, Mercury 3 writes about 850–900 tokens per second. That's roughly 2× Gemini Flash-Lite, 2.8× Claude Haiku and 6× GPT-6 Luna, which lands neatly inside Inception's "4–8× faster" for Luna, and short of it for Haiku. But gpt-oss on Cerebras writes about 2,400–2,500 tokens per second, ~2.7× Mercury, at every output length we tried. Diffusion's advantage is a constant factor, and it doesn't grow on longer outputs.

One nice touch you can feel: short Mercury answers arrive in a single block. Total time (0.30 s) is basically the time to first token (0.29 s). The answer doesn't stream in; it just appears.

Finding 2: In agents, the number of turns matters as much as speed per turn

In the SQL agent, Mercury 3 (low) finishes a whole task in 2.06 seconds at the median. Claude Haiku takes 3.86 s, Gemini 4.43 s, GPT-6 Luna 5.95 s. That's 1.9–2.9× faster than the frontier labs' fast models per task. Diffusion, or at least this product, delivers in the agent loop.

But look at how it gets there:

Claude Haiku is 3.2× slower than Mercury per call, but only 1.9× slower per task, because it solves the problem in about 3.2 turns where Mercury takes 5.1. Haiku thinks less often and better. Meanwhile gpt-oss on Cerebras is 1.7× faster than Mercury per call, but takes 6.3 turns, and ends up only slightly faster per task (1.82 s). At its medium setting it's actually slower than Mercury per task.

Lesson for agent builders: task latency ≈ call latency × number of turns, and the number of turns is a property of the model, not something your speed chart will tell you.

On accuracy, everyone lands within a few points (47.6% to 59.1% correct). Claude Haiku is the most accurate, about 5 points ahead of Mercury 3 when compared on the same questions, but that lead is borderline: its confidence interval reaches down to zero. With 150 questions, the per-model error bars are ±7–8 points, so I wouldn't crown anyone here.

Finding 3: Cheap per token is not cheap per answer

Mercury 3 is priced at $0.10 / $0.35 per million input / output tokens, a 50% launch promo off the regular $0.20 / $0.70. Low per-token prices, but what matters is cost per correct answer:

• Code: Mercury 3 low costs $0.59 per 1,000 passing answers at the promo price ($1.19 regular). GPT-6 Luna with reasoning off costs $0.07. Claude Haiku (thinking off) costs $0.14.
• SQL agent: Mercury 3 instant costs $1.31 per 1,000 correct tasks ($2.62 regular). Luna costs $0.74. gpt-oss on Cerebras and Gemini are the priciest here ($4.83–$6.56), because more turns mean re-sending more context.

Mercury isn't expensive, but it's also not the bargain the per-token sticker suggests, and the promo price won't last.

Finding 4: Mercury 3 sometimes thinks forever

Here's the odd habit. Mercury 3 has reasoning settings, and you'd expect more reasoning to mean better answers. It's the opposite. On code, accuracy drops from 86.0% at low to 82.3% at medium and 81.4% at high.

The reason: at higher settings, Mercury 3 sometimes reasons silently until it hits the 32,768-token output cap and returns nothing at all. That happened on 7.5% of calls at medium and 8.6% at high. Those runaway calls burned 61% of all the output tokens Mercury 3 used in the code test, and pushed its slowest-5% response times past 50 seconds. Giving it 4× more room barely helped: in a small 12-call probe, only one extra call finished. No other model in the code test ran away even once.

My favourite example: an easy-looking problem asking for the "directrix of a parabola," whose single worked example doesn't match the textbook formula. The autoregressive models shrug, write the standard formula, and fail. Mercury 3 at medium and high tries to reconcile the math with the example on all 10 attempts, and thinks until it runs out of budget. Same score as a wrong guess, but at ~32,000 tokens and close to a minute per attempt. Autoregressive models fail by committing; this one fails by not stopping.

Practical advice: use Mercury 3 at low or instant. At low it is also 9 points better than its predecessor Mercury 2.5 at the same setting (86.0% vs 76.9%), and faster (0.83 s vs 1.36 s). Real progress in 20 months.

Surprises that have nothing to do with diffusion

Benchmarking five vendors side by side turns up some gossip.

• OpenAI has a slow front door. Even a free "list models" request took about 0.8–1.2 s of server-side processing at OpenAI, where the same call to the other providers came back in about 15–225 ms. On real requests, OpenAI spent ~0.6 s server-side before doing anything visible. It's not your account tier: paying 2× for OpenAI's Fast mode trimmed only ~0.15 s at the median (it mostly fixes the worst cases).
• The cheapest right answer comes from GPT-6 Luna with reasoning off. Slowest decoder in the test (about 130–160 tokens/s), cheapest per solved problem in both tests.
• Some models really like to talk. Asked for an ~800-line program, Claude Haiku and GPT-6 Luna wrote about 15,000 tokens; gpt-oss wrote about 6,000. Compare speeds in tokens per second, not total seconds.
• Haiku stops exploring when you ask for JSON. With a strict JSON output format switched on, Claude Haiku answered all 150 database questions in one shot, without looking at the database once. It still scored 53.3%.
• Mercury 3 sometimes leaks its own plumbing. In that same strict-JSON mode, it broke the format on 6 of 150 tasks, mostly by appending fragments of its internal tool-call markup after valid JSON. Every other model: zero.
• Cerebras throttled us below its published limit. We kept under its documented 1,000 requests per minute and still collected 765 rate-limit errors in the code test.

So, is it the speed demon as promised?

Back to Inception's claim:

• "More than 1,000 tokens per second": close. We measured ~850–900 visible tokens/s from the client side, over the network, so it includes streaming and network overhead that a server-side number doesn't.
• "4–8× faster than Luna and Haiku": on raw generation speed, about 3× Haiku and 6× Luna. On what you actually wait for, smaller: 1.2–1.5× per coding answer at matched quality, and 1.9–2.9× per agent task.
• "Outperforms Luna and Haiku": ties them when their reasoning is off, trails them when it's on (88.4% and 87.9% vs 86.0% on code). In the SQL agent, all three are within the error bars.
• "Because diffusion": this is where the claim needs a caveat. An ordinary autoregressive model on Cerebras hardware was faster than Mercury 3 in both tests. What we measured is a product (model plus serving stack), not an architecture. Diffusion may well be why Mercury beats the big labs' fast models, but it isn't the only road to speed.

Verdict: Mercury 3 is faster than the big labs' fast models, about as accurate as them with reasoning off, and especially good in agent loops. It's a speed demon, just not the only one in town.

Caveats, honestly

• Products, not architectures. Every comparison includes each vendor's hardware, network and API overhead.
• LiveCodeBench is not contamination-free. All public problems predate the models' training cutoffs, so it serves as a "harder set," equal for everyone, not a fresh one.
• The SQL accuracy numbers are noisy. ±7–8 points per model; only paired differences are worth quoting.
• Promo pricing. Mercury's numbers double at the regular price. Both are shown.
• One machine, one region, one day. Single client host in the US West, standard (non-priority) API tiers, measured October 2026. Your latency will vary.

What I'd test next

• A bigger SQL run (all ~500 scorable BIRD questions) to shrink those error bars.
• Mercury on fast hardware, if Inception ever serves it on Cerebras- or Groq-class chips. That's the clean "diffusion vs. hardware" experiment.
• Code editing: Inception's separate Mercury Edit model does fill-in-the-middle and next-edit prediction, which this study didn't touch.
• Real-time and voice-style workloads, where Mercury's realtime flag might matter (it made no difference for code).
• Controllability: early research suggested diffusion models handle "reverse this" tasks that trip up left-to-right models. Worth a proper probe.

All code, configs, run data summaries, and the scripts that generate every chart here are on GitHub: github.com/zbruceli/mercury3. If you find a bug in my numbers, I'd genuinely like to hear about it.