CrowdStrike has seen customers use our no-code agent-building workspace, Charlotte AI AgentWorks, to build thousands of AI agents. We’ve been fascinated by how they are designed and which ones become top-performing agents invoked hundreds or thousands of times in daily operations. Below are the design properties these agents share.
What top-performing security agents have in common
1. Surgical scope
These agents solve a specific, measurable problem. They do one thing very well, and their output informs broader workflows.
Most organizations start by designing a general-purpose security assistant to address a range of tasks; however, these assistants are challenging to deploy and measure. Even if it can get to a working state, it can easily drift from its objective in dynamic enterprise environments. An agent with a narrow, well-defined job is more likely to become critical infrastructure because its output is predictable and trustworthy enough to measure, test, and build around.
| Example: An agent that owns one type of detection, such as flagging uncommon LOLBIN usage |
2. Multi-agent pipelines
They are also typically cheaper to run, for two structural reasons.
The first is model assignment. In a monolith, the whole task inherits whatever model its single hardest step requires. Every trivial step is then paid for at that rate. Decomposing the task lets each stage run on the optimal model applicable to the agent, and even allows delegating structured tasks to automated scripts or applications that don’t require AI models.
The second is context. A single long-running agent accumulates history, and every subsequent step spends resources to carry it. But an agent pipeline hands each stage only the input that stage needs. A failed stage can also be retried on its own rather than rerunning the entire investigation.
| Example: A threat analysis pipeline built around an orchestrator agent that calls out to a set of narrow specialist agents rather than one broad agent trying to do it all |
3. An explicit specification of what is not allowed
Top-performing agents often include explicit instructions outlining what the agent can and cannot do. This guardrail ensures the agent behavior is bounded. Without it, an agent asked to triage an alert may decide to query adjacent systems, draw conclusions outside its remit, or take actions nobody sanctioned. The guardrail also complements other controls including role-based access, authorized tools, and invocability.
| Example: A remediation recommendation agent’s system prompt states it may propose isolating a host or disabling a user account, but must not execute either action itself, and must not contact any system outside the case record it was given. That guardrail is part of the agent’s configuration. If the agent is invoked with a broader mandate, that expansion has to be a deliberate edit to the agent’s configured instructions, which prevents the agent from deciding it’s capable of more. |
4. A built-in fail-safe
An agent won’t always reach a confident answer. Rather than guess, it should be instructed to escalate ambiguity.
The design response to address this has two parts. One, build a clear escalation path, such as instructing the agent to “flag for analyst review” whenever the agent identifies a circumstance as inconclusive. Additionally, build a checkpoint for the agent, such as a sample of closed cases, or a second agent whose job is to validate the first to catch the cases it doesn’t flag.
| Example: A phishing triage agent’s instructions define three possible outputs: malicious, benign, or needs human review. They explicitly state that “needs human review” is the required output any time the evidence doesn’t clearly support the other two. The agent is measured on accurate “malicious” or “benign” calls, versus closing more cases. |
5. Strictly defined output
The highest invoked agents are part of fully automated pipelines instead of ending up in chat, which requires structured inputs and outputs.
This differs from how most people imagine enterprise AI. The conversational interface makes an agent demonstrable, but it also prevents the agent from becoming part of a defined SecOps workflow. A chat response is consumed by a human who interprets it. A structured verdict, such as a schema with defined fields, bounded lengths, and an explicit decision, can be consumed by a ticketing system, a containment workflow, or the next agent in the pipeline. Structured output allows agents to be chained.
| Example: An email analysis agent has a single output rule: Return the result as a JSON object in an exact, predefined format. That output feeds into the next agent, which reads the fields without requiring additional interpretation. This enables a seamless handoff between agents. |
Building top-performing agents
Start with the right problem
The problem sets up every design decision. The best candidates are high-volume, repeatable tasks with just enough nuance to require reasoning and judgment. Crucially, these can’t be easily addressed with deterministic automation or a script. Existing documented playbooks, or a sequence of steps analysts consistently follow, are great insertion points for an agent.
The task must be narrow enough that its output can be measured and verified. The ability to measure agent impact makes a performance feedback loop possible. It provides a clear way to check whether an output is correct, and whether an edit to the agent helped or hurt.
Choose the right model and test it
Identify the model suited to the task and compare accuracy, speed, and cost across other models. In practice, this can result in an optimal allocation of tools. Matching capability to task optimizes system performance while accounting for costs, latency, and compliance requirements.
Refine instructions
Define what is in scope. Supply context, knowledge files, and organizational context or policies. Specify exceptions, fail-safe procedures, and what to do in inconclusive scenarios.
Optimize output
Work backward from your agent’s handoff in a broader workflow. How does output need to be structured to be consumed by the receiving analyst or tool? Define character limits, required output schema, and tuning required for downstream handoffs.
Implement measurable performance
As part of agent design and governance, implement a means of validating and tracking agent performance — outputting content in a database for human review, conducting a sampling of agent outputs for assessments, or conducting agent testing against a validated data set.
Expand tooling and actions gradually
Measurable agent performance enables security teams to take a phased approach to autonomy, with each stage earned through demonstrated reliability. A common progression begins with summarization: The agent reports; the human decides and acts. Next comes recommendation, where the agent proposes a course of action but the human owns the decision. From there, agents move to approval-gated execution, deciding and staging an action for human approval, and ultimately to full automation, acting independently while humans review later.
CrowdStrike has the right technology and expertise to help with your agentic SOC transformation:
- Read how CrowdStrike customers have been building agents to accelerate security operations using Charlotte AI AgentWorks.
- Watch this on-demand webinar to learn how to get started with building agents within the CrowdStrike Falcon platform.