By Anshul Sawant’s
An AI system can produce an impressive answer while making a poor decision about what should happen next. That gap is the center of Anshul Sawant’s work. As a Principal Applied Scientist at Amazon Ads, Sawant designs agentic systems, AI that can plan and take actions instead of simply returning an answer, along with the evaluation frameworks used to test their behavior. He is concerned that the fluency of modern language models can encourage people to assign them more autonomy than their underlying capabilities justify.
“Language models can be extremely good at executing a task when the operating space is clear,” Sawant says. “The trouble starts when we expect the model to decide how the whole system should be designed.”
Sawant describes the problem as “narrow brilliance.” Large language models can perform sophisticated work inside an assigned responsibility while struggling when asked to determine their own objectives or make broader architectural choices. In his view, that limitation becomes especially important as companies move beyond chatbots toward software expected to plan and act with less direct human involvement.
Current AI systems, Sawant argues, still require substantial engineering around them. Production-ready agents need guardrails that define permitted behavior and evaluation architectures capable of exposing problems before deployment. An impressive demonstration offers limited evidence about how a system will behave once its decisions begin interacting with live data and economic activity.
“An agent can look convincing long before it is reliable,” he says. “Evaluation has to tell you what happens when the system is placed in the environment where you actually intend to use it.”
That problem is tangible in Sawant’s current work. At Amazon Ads, he develops systems that need to understand a company’s brand, audience, and catalog while supporting decisions in live digital advertising environments. The systems operate under strict timing requirements, and their decisions have to remain aligned with the responsibilities they have been assigned.
A correct result in a single test tells Sawant little about whether an agent will behave reliably over time. His work includes building evaluation architectures intended to test autonomous systems before they encounter live data. The objective is to establish whether their behavior remains safe and economically sound under the conditions in which they are expected to operate.
“You need more than a few successful outputs,” Sawant says. “You need a rigorous way to test the behavior of the system and determine whether it stays inside the conditions you designed for it.”
That approach places considerable responsibility back on the engineers surrounding the model. Agentic AI may reduce the amount of human intervention required during individual tasks, but someone still has to determine what the agent is responsible for, how its behavior will be measured, and where its authority ends. The architecture around the model becomes part of the safety mechanism.
Sawant sees particular risk in allowing a language model to move from executor to architect. A model may perform a defined task extremely well while making fundamentally flawed choices about how an entire workflow should operate. His preferred approach assigns the AI specific responsibilities and evaluates those responsibilities directly.
“Open-ended freedom is where these systems get into trouble,” Sawant says. “A model can execute very well inside a defined space and still make poor choices about how that space should be designed.”
The concern reflects a broader misunderstanding Sawant sees in discussions about artificial intelligence. Because modern models can write sophisticated code or generate polished language, users and executives may infer a level of general understanding that the technology does not possess. Sawant argues that convincing output should not be treated as evidence that a system can independently govern the process producing that output.
His Ph.D. in Computer Science from the University of Maryland provides part of his technical foundation, but Sawant’s current approach is rooted in the demands of deploying AI systems that must function repeatedly rather than succeed only under controlled conditions. That production focus has made him skeptical of treating AI as something mysterious that can simply be switched on and trusted.
“People talk about AI as though intelligence appears once you connect the model,” he says. “I think about it as a systems-engineering problem. The model is one part. The architecture around it determines whether the whole thing can operate reliably.”
That philosophy also shapes how Sawant thinks engineers should prepare for the next phase of AI development. He believes practitioners need to understand the architecture surrounding models rather than becoming dependent on sending prompts to increasingly capable systems. Evaluation, deployment discipline, and the ability to reason about an agent’s behavior become more important as the technology receives greater operational responsibility.
The stakes rise when an AI system moves from generating a recommendation to taking an action with economic consequences. An incorrect suggestion can still be reviewed by a person. An agent operating with greater independence may act before someone has the opportunity to intervene, making the quality of the surrounding safeguards more consequential.
Sawant expects agentic systems to improve, but his work suggests that increasing capability should be accompanied by stronger evidence about how those systems behave. Greater responsibility places more pressure on engineers to establish what an agent can reliably handle before expanding its authority.
That standard is less dramatic than the promise of fully autonomous AI, but it addresses the problem Sawant is actually trying to solve. Each additional responsibility given to an agent creates another behavior that must be understood and evaluated. In environments where AI decisions can carry immediate economic consequences, confidence in the model is not enough. Sawant wants the amount of authority an AI receives to be determined by what engineers can demonstrate about its behavior before those decisions become real.