What Is an AI Agent? (And When You Actually Need One)

The word “agent” has quietly become one of the most overloaded terms in tech. A chatbot is called an agent. A script that calls an API twice is called an agent. A swarm of language models arguing with each other? Also an agent.

So before you build anything, it helps to know what the word actually means, and, more importantly, when you need one at all. Because the surprising truth is that the best teams building these systems share one rule: use the simplest thing that works, and only add complexity when it clearly earns its place.

This is your map. By the end you’ll be able to look at any task and say, with confidence, “that’s a plain call,” “that’s a workflow,” or “that genuinely needs an agent.”

Why this matters

Getting this wrong is expensive in a very literal sense.

Reach for a multi-agent system when a single model call would do, and you can burn 15 times more tokens for no extra quality. Reach for a rigid workflow when the task is genuinely open-ended, and your system breaks the moment reality goes off-script.

The cost isn’t only money. Every step up the ladder adds unpredictability, harder debugging, and brand-new ways to fail. Knowing where a task belongs on the ladder is the difference between a system that ships and one that quietly drains your budget.

The three rungs: plain call, workflow, agent

Think of autonomy as a ladder with three rungs. Each one hands more control to the model and takes it away from you.

• A plain LLM call is a single shot. You ask, the model answers, done. A sentiment classifier or a simple chatbot lives here. It may use a tool or pull in a document, but it doesn’t control any flow. This is the most predictable rung.
• A workflow is a fixed script you wrote in code. The model fills in the blanks at each step, but you decided the sequence. “First summarize, then translate, then check the tone” is a workflow. High predictability, because the path is locked.
• An agent flips the control. Here the model “dynamically directs its own processes and tool usage,” in Anthropic’s words. It decides what to do next, which tool to grab, and when it’s finished. Lowest predictability, highest flexibility.

Here’s the cleanest way to feel the difference:

A plain call answers. A workflow follows a script you wrote. An agent decides its own path while it runs.

OpenAI draws the same line, calling an agent a system that “independently accomplishes tasks on your behalf,” one that recognizes when a job is complete and corrects its own course. Google frames it as an application that “achieves a goal by observing the world and acting upon it using tools.” Three different companies, one shared idea: the agent is the thing that steers itself.

And notice: “agentic” is a spectrum, not a switch you flip. Most real systems sit somewhere along this ladder, not neatly at one rung.

The building block underneath it all

Every system on the ladder, workflow or agent, is built from the same Lego brick: the augmented LLM. That’s just a model given three upgrades:

• Retrieval so it can look things up.
• Tools so it can take actions in the world.
• Memory so it can hold onto what matters.

Anthropic, Google, and OpenAI all describe roughly the same three-part anatomy. The model is the brain that decides. Tools are the hands and eyes that act. And an orchestration layer, the nervous system, handles memory, planning, and the loop.

Here’s the part people skip: if one well-built augmented call solves your task, you’re done. A good prompt, the right context, a couple of examples, maybe a retrieval step. Everything beyond this point is complexity you have to justify. Start here every single time.

The agentic loop: the thing that makes an agent an agent

If there’s one mechanic to remember, it’s this: an agent runs in a loop, not a pipeline.

A pipeline goes in a straight line, start to finish. A loop circles back, checking its own work. The loop has four beats:

1. Perceive the task and the current state of things.
2. Plan the next move.
3. Act by calling a tool or running some code.
4. Observe the result, the output, the error, whatever came back.

Then it repeats, looping back to perceive, until the goal is met or a stop condition kicks in (often a maximum number of turns, so it can’t spin forever).

That fourth beat, observe, is the secret ingredient. After every action, the agent gets what Anthropic calls “ground truth from the environment.” It sees what actually happened and adapts. A pipeline fires off its steps blindly; an agent watches and adjusts. That feedback loop is the real line between an agent and everything below it.

ReAct: the simplest recipe for the loop

The cleanest concrete version of this loop comes from a 2022 paper, ReAct: Synergizing Reasoning and Acting in Language Models (Yao et al.). The name is short for Reason + Act, and the idea is almost suspiciously simple.

You interleave three things, over and over:

• Thought - the model reasons in plain language about what to do.
• Action - it takes a step, like querying an API.
• Observation - it reads back the result.

Why does mixing reasoning and acting work so well? Because each one fixes the other’s weakness. Pure reasoning (“chain of thought”) tends to drift into confident nonsense, since nothing checks it against reality. ReAct grounds every thought in a real observation from the world, which the paper showed cuts down hallucination. On interactive benchmarks like ALFWorld and WebShop, it beat the baselines by wide margins.

There’s a bonus: because the agent writes out its thinking, you can read the trail and understand why it did what it did. That transparency is worth a lot when something goes wrong.

The four-beat loop is the spine of every agent. ReAct is the simplest way to build that spine.

What actually makes something “agentic”

When you strip away the marketing, four properties separate a real agent from a fancy LLM call. The more a system has, the more agentic it is.

1. Dynamic control flow. The model, not your code, decides the next step and how many steps there are. You literally cannot predict the path in advance.
2. Tool use. It can act on the world, not just produce text.
3. Environment feedback. It observes the results of its actions and adapts. This is the closed loop again.
4. Iteration toward a goal. It keeps going in a loop until the goal is met, instead of answering once.

A plain classifier has none of these. A retrieval call has tools but no loop. A workflow has tools, feedback, and iteration, but its steps are still nailed down in code. A true agent has all four.

And if you want the single property that promotes a humble workflow into a real agent, it’s the first one: dynamic control flow. The moment the model chooses the steps at runtime instead of following your script, you’ve crossed the line.

Five workflow patterns to try before you reach for an agent

Here’s a thought that will save you a lot of grief: most “I need an agent” problems are actually workflows in disguise. Workflows are cheaper, more predictable, and far easier to debug. Anthropic catalogs five patterns that cover an enormous range of tasks.

• Prompt chaining. Break the task into a fixed sequence of calls, each feeding the next, with simple checks (“gates”) between steps. Great when the task splits cleanly and you’ll trade a little speed for accuracy.
• Routing. Classify the input first, then send it down a specialized path. Perfect when different kinds of input deserve different handling, like sorting support tickets by type.
• Parallelization. Run pieces at the same time. Either split independent subtasks (“sectioning”) or run the same task several times and combine the answers (“voting”) for higher confidence.
• Orchestrator–workers. A lead model breaks the task into pieces at runtime, hands them to worker models, and stitches the results back together. Use it when you can’t predict the subtasks ahead of time.
• Evaluator–optimizer. One model writes, a second critiques, and they loop until the work passes. Ideal when you have clear quality criteria and revision genuinely helps.

Notice that one pattern is sneaky. Orchestrator–workers is not the same as parallelization, even though both involve multiple workers. In parallelization, you fix the subtasks in code. In orchestrator–workers, the lead model decides the subtasks while it runs. That runtime decision is dynamic control flow showing up again, which is exactly why this pattern is the bridge from “workflow” to “true agent.” Keep an eye on it.

Single agent versus multi-agent

So you’ve decided you genuinely need an agent. The next question, and the one people rush, is whether you need more than one.

What multiple agents actually buy you

There are real wins here, and they’re worth knowing:

• Breadth. Anthropic’s multi-agent research system, where a lead researcher spawns three to five subagents to explore in parallel, scored a 90% improvement over a single agent on their internal research benchmark. It shines on questions that fan out in many independent directions at once.
• More room to think. Each subagent has its own context window, so the system can chew through information that would never fit in a single one.
• Speed. Running tools in parallel cut research time by up to 90% on complex queries.
• Cleaner separation. Distinct agents can map neatly onto distinct concerns.

What it costs you

Now the bill, because it’s steep and easy to ignore.

• Tokens, lots of them. A single agent already uses roughly 4x the tokens of a chat. A multi-agent system uses around 15x. In one analysis, token usage alone explained about 80% of the performance difference between systems. You are mostly paying for thinking.
• A hard economic floor. Multi-agent only makes sense when the task is valuable enough to justify that token bill. Below that line, don’t.
• Coordination is clumsy. Today’s models aren’t great at delegating to each other in real time. Lead agents often wait for each batch of workers to finish, creating bottlenecks.
• Errors compound. These systems run a long time across many tool calls. A small slip early can snowball into a wrecked result.
• They’re non-deterministic. Run the same prompt twice and you may get two different paths. That makes debugging genuinely hard.

When multiple agents are simply the wrong tool

Some tasks fight against the multi-agent shape. As Anthropic puts it plainly, domains where every agent needs to share the same context, or where the agents have many dependencies on each other, are a poor fit today.

The clearest example may surprise you: most coding tasks. Code has tightly woven dependencies and far fewer truly parallel pieces than research does. A good fit looks like the opposite, high-value work that splits into independent parallel threads, with information too big for one context window.

The independent reality check

It’s not just the vendors warning you. A Berkeley study, Why Do Multi-Agent LLM Systems Fail? (the MAST project), combed through more than 1,600 annotated traces across seven popular frameworks and found that “performance gains on popular benchmarks are often minimal.”

They catalogued 14 distinct failure modes in three families: flawed system design (agents repeating steps, ignoring when to stop, disobeying the spec), agents talking past each other (which accounted for roughly a third of failures), and weak verification of the final output.

The lesson is blunt: every agent you add brings a fresh set of ways to fail, stacked on top of all the single-agent ones. More agents is not free quality.

Common misconceptions

A few myths are worth clearing up, because they drive a lot of bad decisions.

• “Agent just means LLM with tools.” Not quite. Tools are necessary but not sufficient. Without dynamic control flow, where the model picks its own steps, you have a workflow, not an agent.
• “More agents means better results.” Often the reverse. Without the right task and careful coordination, extra agents add cost and failure modes while gains stay “minimal.”
• “An agent is just a longer chain of prompts.” A chain runs blind from start to finish. An agent observes each result and changes course. The feedback loop is the whole point.
• “Multi-agent is the advanced, serious choice.” The genuinely advanced move is using the simplest thing that works. Sometimes that’s a multi-agent system. Far more often it isn’t.

How to use this: the decision ladder

Here’s the whole framework as a checklist. Walk down it in order, and stop the moment something works. The burden of proof is always on the added complexity.

1. Try a single, well-engineered LLM call first. Good prompt, retrieval, a few examples. If it solves the task, ship it. As Anthropic notes, the right answer is often “not building agentic systems at all.”
2. Are the steps fixed and knowable? Build a workflow. Use chaining, routing, or parallelization. It’s predictable, cheaper, and easy to debug.
3. Is the path genuinely unpredictable, needing decisions at runtime? Build a single agent with tools in a loop. Push this as far as it will go before doing anything fancier.
4. Is that single agent overloaded? The warning signs are more than roughly 15 distinct tools, several tools that overlap and confuse it, or prompts swelling with if-this-then-that branches. If not, stay single-agent and improve it.
5. Only if it’s overloaded AND the task is high-value, parallelizable, and bigger than one context window, go multi-agent. Fail any of those and the answer is to fix the single agent, not split it.

A quick gut check from OpenAI on whether a problem even deserves an agent: reach for one when the task involves nuanced judgment (like approving refunds), rules too tangled to maintain, or messy unstructured data like free-form documents and conversation. If a problem hits none of these, a plain rules-based program probably beats any agent.

And once you do split, there are two clean shapes to choose from:

• Manager pattern. One central agent stays in charge, calls the others like tools, and owns the conversation. Best when you want a single controller synthesizing everything.
• Handoff pattern. Agents pass control to one another, each able to take over the user interaction. Best when no single agent needs to hold the reins.

One more practical tip that punches above its weight: invest in your tools as much as your prompts. Anthropic found teams pour effort into wording prompts while neglecting tool design. Write clear tool descriptions with examples and edge cases, design tools so they’re hard to misuse, and test them hard. In their system, a single agent dedicated to rewriting confusing tool descriptions cut later task-completion time by 40%.

Conclusion

If you carry one idea out of this, make it the ladder: plain call, then workflow, then single agent, then multi-agent, and never climb a rung you can’t justify. The property that moves you up is dynamic control flow. The force that should hold you back is the roughly 15x token cost and the pile of new failure modes that arrive with every extra agent.

The teams who get this right aren’t the ones with the most agents. They’re the ones who reach for the simplest tool that does the job, and resist the pull toward clever.

Which raises the obvious next question. Once you’ve decided a single agent really is the right call, how do you actually build one that stays reliable across dozens of tool calls without quietly going off the rails? That’s where the real craft begins, and it’s exactly where we head next.