Why AI agents might not be right for you

There’s no question that 2026 is the year of the artificial intelligence agent. AI agents are remarkably ubiquitous, as people use them to write code, communicate with customers, identify and resolve issues in their information technology environments, and execute complex tasks within enterprise environments.

It seems there’s nothing that agents cannot do. And yet, the risks inherent in agentic behavior are in the news almost as much as their success stories. Horror stories of agents going off the rails, breaking out of sandboxes and hacking other companies and government agencies fill the airwaves.

Your organization’s AI strategy may depend on AI agents, but given the risks, should you really be depending on them for your company’s success?

Nondeterministic behavior: Agents’ greatest strength and greatest weakness

Any decision to deploy agents depends upon understanding their nondeterministic behavior.

Today’s agents generally depend upon large language models. LLMs take human language inputs and produce human language outputs by predicting the next word in a sequence.

Given the way the models are trained, repeating a given input may produce quite different outputs. In other words, LLM-based agents are inherently nondeterministic.

This nondeterminism gives agents their unique power, as without it, agents wouldn’t have agency: the ability to make decisions based upon available information to take different actions to accomplish the goals set out for them.

Software without agency aren’t agents, by definition – although there are plenty of examples of “agent-washing” where people call deterministic software agents.

There are other examples of nondeterministic software that don’t have the agency that LLMs provide, namely software based upon stochastic policy-based nondeterminism. Common examples of such stochastic software include reinforcement learning and Monte Carlo simulations: software that makes decisions based upon randomization.

Other than such limited examples, however, the vast majority of software ever written before the agentic wave was deterministic: For given inputs, the software always does the same thing.

Such predictable behavior, of course, is our motivation for building software in the first place. Whether we’re processing bank transactions, running payroll, assigning seats in airplanes, or any of millions of other tasks we’ve used computers for over the last century, predictable behavior is what we want.

So why do we want AI agents’ unpredictable behavior now?

What AI agents are really good for

Deterministic software is best when there is a right answer, and you want to find it. Think of running payroll: The goal is to ensure everyone gets the correct pay – end of story.

In contrast, LLM-based nondeterministic reasoning makes the most sense when you want the best result given a range of different or changing inputs, even though there may not be a single right answer.

Examples include weather-dependent planning, incident mitigation in complex information technology environments, and business forecasts based on multiple inputs, to name a few.

In such situations, not only is there no single correct answer, but the answer the agents do come up with may or may not be the best result. There is always a chance that an agent will give a poor answer or take an undesirable action.

If you don’t want to take such a gamble, then don’t use agents.

When using AI agents is a bad idea

There are several situations where using AI agents is a bad idea. Here are some of the most significant:

• When deterministic software meets your needs. Not only do agents misbehave on occasion, but they can also be expensive to run. If more traditional, deterministic software will suffice, then agents are generally a bad choice.
• When every decision or action must be exactly correct. Don’t let that agent touch your payroll.
• When agentic misbehavior can cause serious harm. The chance of an agent misbehaving might be low, but if such misbehavior leads to a dead hospital patient, a crashing airplane or an exploding oil rig, then even a low probability of an agent going off the rails isn’t worth the risk.
• When agents are too expensive given the business benefits you’re looking to achieve. Agents may make several LLM calls, retry various actions, and invoke expensive tools on their way to completing their goals. If the cost of such behavior outweighs the business benefit you’re expecting, then agents don’t make economic sense.
• When human qualities are essential to achieve your goals. LLMs (and hence, agents) are good at simulating human characteristics such as empathy, creativity, insight and understanding, but all they are really doing is mimicking human behavior. When your goal requires such human capabilities, don’t be fooled: Anthropomorphizing agentic behavior will always backfire.

The anthropomorphism of agentic behavior (and AI generally) is a subtle and poorly understood risk – but perhaps the most pernicious of them all. Believing that bots can actually think and feel like a human can lead to increased suicide risk, overdependence on technology, and other dangerous, counterproductive behavior. Be careful.

How to decide when to use agents

To determine whether the decision to deploy agents is the best one in a particular situation, take a page out of the site reliability engineering or SRE playbook: Calculate the error budget.

The error budget represents the acceptable cost of agent misbehavior, given all relevant factors, expressed as a percentage. For example, you might calculate that your error budget is 5%, meaning that you are OK with agents misbehaving 5% of the time, given how much more it would cost you to reduce this number.

If you calculate your error budget for an agent is zero, then you have just concluded that you shouldn’t deploy that agent at all. Remember, there’s always a chance that an agent will misbehave.

Once you have your error budget, then conduct a simple exercise in decision theory: When the error budget (probability of failure) multiplied by the all-in cost of failure is greater than the probability of a successful outcome multiplied by the quantified business benefit of that outcome, then you shouldn’t deploy the agent.

In other words, the business benefit you expect to gain should your agents behave properly should exceed the total cost of failure.

There is one more factor to consider, however. Remember that you’re basing this calculation on estimates, any or all of which may be incorrect.

As a result, your expected business benefit should exceed your all-in cost of agentic failure by a wide margin – where just how wide depends upon your overall appetite for risk.

The Intellyx take

A prevalent trend in the AI marketplace today is to respond to the risks inherent in agentic AI with technologies to mitigate those risks. The theory is that if only we had better governance and security software, then we’d be able to rein in our agents sufficiently to deploy them with impunity.

However, there is a flaw in that reasoning. The thrust of much of the innovation in agentic governance is to restrict the unpredictable nature of agents to keep them in line – despite the fact that it is that very unpredictability that gives agents their agency.

In other words, we risk killing the Golden Goose: LLMs give us these enormously powerful agents, and in response, we do what we can to hobble them to the extent that they are no longer nearly as useful as we had hoped.

I envision a different future. Once all the hype around AI agents has simmered down and the technology has matured, my prediction is that we’ll find that agents are best used sparingly.

Deterministic software has met our needs for decades, after all. Do we really want agents to take over? I have my doubts.

Jason Bloomberg is founder and managing director of Intellyx, which advises business leaders and technology vendors on their digital transformation strategies. He wrote this article for SiliconANGLE. All of Intellyx’s content is 100% human generated, including this article.

Image: Craiyon