For the past half century or more, the teams running enterprise software stacks have nurtured a vast ecosystem of tools to keep software humming along. They’ve built CI/CD pipelines, elaborate test suites, code repositories, and tools galore, all with the goal of keeping the systems up and running.
Now AI is throwing a wrench into the whole endeavor. Software is now more complex and sometimes mysteriously unpredictable. Expectations are sky high, but no one is certain what to expect in good conscience.
Luckily, dozens of companies are rushing to build new tools designed to keep agentic stacks on the straight and narrow, delivering value to everyone. The field is rapidly evolving, probably having changed since you started reading this article.
Some of the tools involved in the emerging agent stack aren’t that different from what enterprise teams already have in store. In the end, LLMs are just piles of bits that need to be loaded into a processing chip, just like old-fashioned software. Many of the old tools that watch for overloaded nodes or overpacked disk drives are just as valuable as ever, perhaps more so because LLMs chew through storage even faster. And some of the companies tackling these problems are the same steadfast suppliers who have earned the trust of your DevOps teams; they’ve just extended their own offerings to handle LLMs and AI.
Other challenges, however, are quite different. For example, LLMs are terrifyingly non-deterministic, a real challenge for belts-and-suspenders engineers who traditionally tend the software. Tool vendors are building entirely new systems to handle these somewhat random deviations so that traditional DevOps teams — and their C-suites — don’t have heart attacks.
The field is also rapidly expanding in wildly divergent ways. Some vendors are building tools for developers who are creating new LLMs. Others are concentrating on the challenges that come from worrying about an LLM, day in and day out, in large scale production.
We’ve compiled lists introducing the most intriguing companies developing tools for this rapidly altering space, including tools aimed at AI orchestration; AI observability, aka AgentOps; AI evaluation and benchmarking; and AI trust, guardrail, and governance. We’ve even thrown in some vibe-coding tools for citizen development and AI workflow tools for business processes.
While it was relatively easy to split the field into distinct lists, that process will only grow more challenging because the tools — and categories — are starting to overlap. As vendors evolve their strategies, this overlap will grow further, with a variety of entries tackling a wider range of challenges, vying for a greater percentage of CIOs’ AI middleware and operational budgets.
To help you make sense of this complex tool landscape, here are the most common categories vendors are using to identify what their products do, each of which maps to what are becoming the key components IT needs to keep their AI and agentic stacks productive and humming.
Observability
Vendors often use the term “observability” to refer to the most basic of data collection work — that is, testing to make sure the various machines are running, the RAM isn’t too full, latency is acceptable, and the logs aren’t filled with error codes.
Watching these metrics isn’t much different from caring for an old-school collection of microservices, and often the tools aren’t much different, either. Many of the most popular tools are the same dominant market leaders that have been extended to collect data from the various GPUs, too.
Leading vendors: DataDog, Dynatrace, New Relic, Splunk
Upstarts: Ariz, LangSmith, Braintrust, Galileo
Open-source alternatives: OpenTelemetry, Phoenix, Langfuse
Cost engineering
Many AI pipelines are often more complex than regular microservice architectures, and that means they need someone to carefully watch the bill. Plus, as attention to AI ROI rises, capturing spend is essential — especially given how quickly AI tokens can add up.
Cost engineering tools can integrate information from various portals like OpenRouter and compare it against internal hardware. Sometimes breaking down these comparisons is difficult because the right way to allocate costs to each user isn’t as obvious, even though they’re often just one number. These accounting operations tools will become even more important as LLM bills grow.
Leading vendors: CloudZero, Finout
Upstarts: Amnic, Bifrost, Portkey
Open-source alternatives: OpenCost, Langfuse
AgentOps
When metrics and telemetry begin to focus on the LLM, the word “AgentOps” starts to appear. It’s just like DevOps but it’s concerned with watching token counts, tracking the performance of RAG databases, and collecting traces for debugging purposes.
When the work is truly agentic, AgentOps must also juggle the state that agents generate while looping through their various activities. Agents aren’t just a one-and-done call to an LLM. They’re software packages that need to store information as they work toward a final answer.
This job of an AgentsOps team gets trickier because these areas are where the models often behave non-deterministically. Is some model hallucinating? Are an agent’s prompt chains running to completion? The tools in this space track all these metrics, often in a dashboard with nice graphs that make problems stand out.
Leading vendors: Databricks, Datadog, Dynatrace
Upstarts: Arize, Braintrust, Comet
Open-source alternatives: Langfus, LlamaIndex
Evaluation
The buzzword “evaluation” starts to appear alongside tools that are used in the earlier stages of development, when various models and prompt combinations need to be tested again and again.
In many cases, these tools use another model to evaluate the quality of the answer from the first model, as well as to keep an eye out for potential hallucinations or other failures. In some cases, the tools query humans who sit in the evaluation loop to give their opinions of how well the model is performing. Good tools can watch the agents continuously and reroute any sketchy behavior to the human-in-the-loop.
Many of the vendors use the word “evaluation” to describe measuring just how well a model is performing for a particular business application. That is, how well it’s working for what your boss wants it to do.
Leading vendors: Databricks, SuperAnnotate, Scale AI
Upstarts: Braintrust, Galileo AI, MaximAI
Open-source alternatives: Langfuse, Prompfoo, Tagas
Benchmarking
The word “benchmarking” is similar to “evaluation” in AI vendor literature, but it is usually found more in the tools used by the creators of the models. There are a number of public benchmarks filled with standard queries, and the model developers rank themselves on how well they do solving these problems. This often generates a raw score that is then compared to another’s raw number. Some teams create their own benchmarking problem sets and use them during evaluation to assess how well models might meet their needs.
Leading vendors: Hugging Face, Databricks
Upstarts: Reka AI, Patronus AI
Open-source alternatives: Promptfoo, DeepEval
Trust
The tools that wear the “trust” benchmark are often meant to help evaluate how well a model is protecting the needs of any business that is deploying it. That is, can they trust it to deliver accurate and useful answers without hallucinations, data leakage, inappropriate behavior, or worse? Prompts with malicious instructions are especially tricky to detect. Some enterprises want a trust score that measures the consistency of the answers. Overall, tools that target the “trust” metric are often utilized more during development and early deployment.
Leading vendors: Azure AI Content Safety, Bedrock Guardrails, Vertex AI Search
Upstarts: Patronus AI, Arthur AI
Open-source alternatives: NeMo Guardrails, Llama Guard
Guardrails
The word “guardrails” is usually applied to tools that are part of the deployment stack. That is, they sit as watchdogs and make sure the LLM doesn’t spin out of control. They also ensure the model is trustworthy, but they’re more tuned to operating in real-time than those aimed at trust, as outlined above.
These tools are often filled with fixed rules, such as scanning for social security numbers or watching for noxious words. The challenges get trickier because agents are executing software, and that means the guardrails must do more than analyze outputs. They also need to manage the execution environment, often running in a sandbox, and watch for exploits and other malicious behavior. They need to validate and double-check not just code, but the OS-level commands for manipulating files and processes.
Leading vendors: Azure AI Content Safety, Bedrock Guardrails, Vertex AI Search
Upstarts: Patronus AI, Arthur AI
Open-source alternatives: NeMo Guardrails, Llama Guard
Context
Many systems rely on a vector database to store embedded data to chunks of trustworthy documents. These RAG databases need much of the same care as regular databases, and the tools for their care are often similar. PostgreSQL, for example, has a vector addition that needs the same kind of scrutiny from a database administrator as a regular PostgreSQL database.
Leading vendors: MongoDB, Elasticsearch
Upstarts: Pinecone, Weaviate
Open-source alternatives: Milvus, PostgreSQL
Gateway routing
Increasingly, AI systems employ models in constellations, and a gateway router is what examines a prompt to determine which model or agent is best for answering the query. Moreover, as agents undertake their work, the router can distribute the agent’s various subcalls to the fastest, cheapest, or most accurate LLM or subagent.
Watching over this stage in the pipeline requires a different set of intelligence because the questions can often involve both applicability and cost. Sometimes this layer must balance the extra cost versus the need for a better answer.
Leading vendors: Cloudflare, AWS, Google Cloud, Azure
Upstarts: OpenRouter, ModelsLab, Respan
Open-source alternatives: LangChain, Bifrost, LiteLLM
Access control
Who gets to use the model? Who is paying? Many access control tools are the same ones that were part of standard APIs or web services. Is there an API key? Is the account below its budget? The tools watch for overspending because both the customer and the boss want to make sure the money keeps flowing and no tokens are wasted.
Leading vendors: Auth0, Apigee
Upstarts: Amnic, Portfoundry
Open-source alternatives: LiteLLM, Bifrost
Compliance
Some industries have explicit compliance requirements dictated by law. Some have their own internal controls. Tools devoted to compliance are often a subset of the ones devoted to guardrails. But they’re often treated separately from guardrail tools because they’re more likely to be of interest to lawyers and that makes the stakes higher.
Leading vendors: ServiceNow, IBM
Upstarts: VerifyWise, Norm AI
Open-source alternatives: Aether, Giskard