In the AI SDLC, writing code stopped being the constraint and verifying it took its place. Agents now write code faster than any review or test process was built to absorb, and none of them can tell you whether the result is any good. The auditor cannot be the creator: whoever writes the test suite only tests what they already thought of, and that holds for a coding agent as much as for a person.
The bottleneck moved from writing code to verifying it
Every process we built assumed writing code was the hard part, because it was. Tickets, handoffs, review gates, and quality assurance (QA) all existed to turn scarce engineering hours into working product.
The scarce hours are gone, and the telemetry shows where the pressure went. Faros AI's AI Engineering Report 2026 (22,000 developers, more than 4,000 teams) found pull requests merging with no review at all, human or agentic, up 31.3%, and median time in review up 441.5%. Reviewers cannot keep pace with the volume, so more code reaches production that nobody looked at.
Watch what the old model does with that pressure and you see a loop that never resolves: ship, find the problems by hand, write a test for what you found, watch the test break on the next refactor, repeat. Hiring more QA does not close the loop. It staffs it.
So the constraint moved to verification and judgment. The question stopped being "can we build it" and became "does it work, does it hold together, and is it good enough to ship." Those are product questions, and they do not have scripted answers (we make the full case in Past the Bottleneck). For the mechanics of the cycle itself, see the AI-driven SDLC explained; this piece stays on the strategy.
How fast does feedback land on a pull request?
Leaders ask this one first because they feel it daily. If a check takes a day to come back, developers merge around it.
The instinct is to run the whole suite faster. That only speeds up the old model. What should run is scoped to what the pull request (PR) changed, and risk-weighted: a docs-only change gets little or nothing; the payment flow gets everything. The agent picks up the PR and its preview deployment, tests through the real user interface, and posts pass or fail before a human opens the review.
On QA.tech's PR testing that means connecting GitHub for automatic reviews, or tagging @qa.tech in a comment, with the verdict on the PR in minutes. You can require the check in branch protection as a merge gate.
A real verdict on a public example PR: the change is classified, the coverage is stated, the one failure is named with its evidence, and the paths not covered are listed.
Fast feedback only counts if you trust it and do not drown in maintenance.
The cost of test maintenance
A Playwright, Cypress, or Selenium suite costs most after it's written. A selector changes, an animation runs long, and a green test goes red with no bug behind it. Brittle by design: the test was bound to details a user never sees (the hidden tax adds up).
An agent that works from a goal instead of a script removes most of that at the source, because it finds elements by what they are and never depended on the selector (which is also why self-healing scripts miss the point). That solves maintenance but not coverage, and coverage is the bigger problem.
Explicit assertions only catch the bugs you already imagined, so a suite's ceiling is whatever its author thought of on the day. It is trivial to write a Playwright script that verifies you land on /dashboard after login while every metric on that dashboard is broken. What you wrote was "verify the user is at /dashboard." The name for that gap is false coverage.
That reframe survives the rebuttal the maintenance argument does not. An agent writes my tests now, so maintenance is cheap. True, and it solves half of it. The agent is still only testing what it thought of, and if it also wrote the code, it is the last thing you should ask about the code's quality.
The field data says the same. In Stack Overflow's 2025 survey, 66% of developers using AI tools named "almost right, but not quite" solutions as their biggest frustration: code that passes a glance and fails in use. In METR's randomized trial with experienced open-source maintainers, developers expected AI to make them about 24% faster and measured 19% slower, and still believed afterwards that it had sped them up. Self-assessment of AI-assisted work is unreliable even when the assessor knows the codebase well.
We hit it ourselves this summer. A feature spec named a known bug to avoid; the same warning sat in a code comment next to the function. The coding agent read both and shipped the bug. A separate testing agent caught it by clicking through the preview. Asked to retrospect, the coding agent wrote: "I had the answer written down twice and I shipped the bug anyway." Nothing in that loop was exotic. The producer optimized for a green run, and the only thing that disagreed was the one component that never saw the diff.
Where this sits in the AI SDLC
Here is a ladder for the AI software development life cycle (SDLC). Find your rung, then look at the one above it.
- No AI in the loop. Humans write and test everything by hand.
- AI-assisted coding. Autocomplete helps; testing has not caught up.
- Agentic coding. Agents ship PRs faster than humans can review them.
- Agentic PR testing. Every PR is tested in a live preview.
- Agentic post-merge. Verification follows the code after it merges.
- Fully agentic SDLC. Development and test agents pair locally, over the Model Context Protocol (MCP).
Most teams shipping AI-written code sit at stage 3, the gap the Faros numbers measure. Stage 4 closes the PR gap; stage 5 covers teams without preview environments, running the same review against staging after merge; stage 6 puts the quality signal inside Cursor or Claude Code while the code is written. Many teams believe they already sit at stage 6; the next section is what that takes.
The ladder is about maturity. The pipeline below is about placement: the same verification layer attaches at whichever stage your team already has. Each dark box names the rung it serves, so a team at stage 2 starts with scheduled test plans on staging and smoke checks in production, a team at stage 3 or 4 attaches to the pull request, and a team at stage 6 runs it before the diff leaves the machine. Nothing in the top row changes. The findings land where the team already works, and the fix ships on the next commit.
Top row: a typical pipeline and the tools in it. Bottom row: where verification attaches, and which rung of the ladder each entry point serves.
Whichever rung, the suite shape is the same: a small scheduled set of must-never-break flows, and verification generated per change on every PR. Suite size is a metric from the scripted era. When tests are written for the change, hoarded tests are inventory to prune, and a suite of a few hundred at a 1% flake rate fails almost every run for reasons unrelated to the change.
Skip the DIY
You can build this yourself, and on day one it looks done: wire a coding agent to Playwright and CI and it's green the same afternoon. That reads as success, which is the problem, because the limits show up at month three: who maintains it, who looks when it goes red, and what happens when the same agent wrote both the code and the test that vouches for it.
The easy cases are real: simple flows, a small surface, a team that wants to own its tests. Sometimes building it yourself is the right answer. The durable flaw is the one from two sections back: you only test what you already thought of, whether a person or an agent writes the assertions.
Four things a local agent cannot do on its own, and none of them is about model quality.
It does not compound. One PR should teach the system something that re-runs whenever a change touches the same area; a local agent starts from zero every run and keeps nothing. It does not look. Code says what should render; only the rendered screen says what did, and an agent that never opens the page is reasoning from the spec, which is the thing under test. It does not orchestrate. One to three browsers, the login chain redone every run, your own machine saturated and your push blocked until it finishes, against parallel browsers, resumable runs, and logins and test data reused across whole flows. And it does not standardize. Every developer's local agent behaves a little differently, so nothing is consistent across the team and nothing accumulates.
So when the objection is "I already do this with Codex": you do, for the easy cases. The hard part is everything around the agent: the application and environment model, the reusable logins and data, the loop that turns a missing permission into configuration instead of a dead end, and the feedback that compounds into rules. Reproducing that is a harness, and a harness takes months.
If you would rather not build the harness, QA.tech runs the agentic layer and puts its agents in the same editor your coding agent lives in, over MCP. It won't catch everything. What it takes off your plate is the harness. Start with the free field guide Past the Bottleneck; to see it on your own repository, book a demo.
QA, in its old form, is dead. In its place comes something more valuable: verification that sits outside the thing that wrote the code and remembers what it learned. A real-time system for product quality.
Vilhelm von Ehrenheim, QA.tech's co-founder and Chief AI Officer, is working through this live on 22 October in Nobody Reads the Code Anymore: how agents can run verification that keeps up with AI-written code, map the app and score the blast radius of each change, and catch bugs before they reach customers. If your review queue is at an all-time high, the session is the practical version of this article.
FAQ
Why does scripted test automation break at AI speed? Scripts bind to selectors and fixed paths, so AI-generated changes turn them red even when nothing broke, and they only ever check what their author already imagined.
Can non-deterministic AI tests be trusted? Determinism never made a test trustworthy; a deterministic test reliably checks the one thing its author imagined. Trust comes from auditable evidence: video of the run, network and console logs, and a per-step trace stating what was checked and what it saw. You verify your verifiers the same way you verify anything else.
What is the AI SDLC? The software development life cycle once AI tools assist coding, testing, and deployment, which shifts the main constraint from writing code to verifying it.
Can I just point a coding agent at my app? For simple flows, yes. What you do not get is the harness around it: compounding memory, a visual check on the rendered screen, parallel orchestration, and one standard shared across the team.
What is a quality gate for AI-generated code? A required check on the pull request that tests the change against a live preview and posts a verdict with evidence before a human reviews it. Docs-only changes pass through; payment flows get the full treatment.