Designing Agent Workflows

Almost every agent demo works. That's the problem. It works once, on stage, on the happy path, and the room claps. Then it goes to production and falls over the first time reality deviates from the script, and the team spends the next quarter discovering that the demo was the easy 10 percent.

The patterns that close that gap are not exciting. The single most useful piece of guidance in the field, from Anthropic's widely-read "Building Effective Agents," is a deflation: most of the time, you should not build an agent at all. You should build a workflow. Start with the simplest thing that works, and only add autonomy when the problem actually demands it.

That advice sounds anticlimactic. It is also the whole game. So let me lay out the patterns that work, in the order you should reach for them, across the three things every agent has to get right: how it acts, how it remembers, and how it's organized.

Workflow first, agent only if you must

The first design decision is the one people skip: workflow or agent?

The distinction is clean. In a workflow, you define the path. The model fills in steps along a route you laid out in code: do this, then route to one of these branches, then combine the results. In an agent, the model decides the path itself, choosing what to do next based on what it sees. Anthropic's framing names the workflow patterns plainly: chain prompts in sequence, route an input to the right handler, run steps in parallel and combine them, or set up an orchestrator that hands subtasks to workers.

Workflows are predictable, cheaper, and easier to debug, because you can see the path. Agents are flexible and powerful and risky, because you can't. The common mistake is reaching for a fully autonomous agent when a three-step workflow would have done the job with a tenth of the failure modes. Autonomy is a cost you pay for flexibility you might not need. Spend it deliberately.

The rule of thumb that holds up: if you can write down the steps, write down the steps. Save the agent for the cases where the steps genuinely can't be known in advance.

How it acts: tools, kept few

When you do need an agent, the foundational pattern for how it acts has been stable since the ReAct work in 2022: the model reasons about what to do, takes an action through a tool, observes the result, and reasons again. Think, act, look, repeat. Almost every agent you've used runs some version of that loop.

Which means the real design work isn't the prompt. It's the tools. An agent is only as capable as the actions you give it and only as reliable as the descriptions of those actions. And here the instinct to be generous backfires. Hand an agent fifty tools and you've done two bad things at once: you've burned context describing tools it won't use, and you've made every decision harder, because now it has to choose among fifty options at every step.

The agents that work tend to have a small number of well-chosen, well-described tools. Each tool does something clear. The descriptions read like good documentation, because to the model they are the documentation. If your agent is flailing, the fix is usually not a smarter model or a longer prompt. It's fewer, sharper tools.

How it remembers: think out loud, on paper

An agent that takes more than a few steps has a memory problem, and it's the same problem from every other piece I've written here: context is finite, and stuffing everything into the window makes the model worse, not better.

The pattern that solves it is almost embarrassingly simple. Have the agent write its plan to a file, then work against that file, updating it as it goes. Plan first, on paper, then execute. The plan becomes external memory the agent can re-read instead of holding everything in its head, and it survives the context resets that happen on any long task. It has a second benefit that matters even more in production: you can read it. A plan written to a file is a window into what the agent thinks it's doing, which is the difference between an agent you can supervise and one you just have to trust.

This is the same instinct as Reflexion, the idea that an agent should be able to look back at what it did and adjust. Memory in an agent isn't a vector database you bolt on. Mostly it's a scratchpad the agent keeps, externalized so it doesn't rot and so a human can look over its shoulder.

How it's organized: one agent, or many?

This is the live debate, and it's worth getting right because the wrong answer is expensive.

One camp argues for keeping it simple and single-threaded. Cognition made this case directly in a piece titled, with no ambiguity, "Don't Build Multi-Agents." Their argument is about context: when you split work across several agents, they make decisions in isolation and then can't reconcile them, because they never shared the full picture. A single agent carrying one coherent thread of context beats a committee of agents talking past each other.

The other camp points at real systems that went multi-agent and won. Anthropic described building a research feature where an orchestrator farms a question out to parallel sub-agents, each exploring a different angle, then synthesizes. For breadth-first work, the parallelism is the point.

Both are right, and the field mostly talks past itself by not naming the difference. Sequential, single-agent wins for exploratory work, where you don't know the answer and each step has to learn from the last. Parallel, multi-agent wins for work that genuinely splits into independent pieces: research this angle, check that file, draft this section. The failure mode everyone warns about, the swarm, is several agents racing on the same task, where they duplicate effort, make conflicting choices, and burn tokens generating noise. The good multi-agent setups are never that. They're an orchestrator handing out disjoint jobs, with one thin coordinator holding the thread.

So the question isn't "how many agents." It's "does this work split cleanly into independent pieces?" If yes, an orchestrator with specialists. If no, one agent in a loop. Almost never a swarm.

The part everyone skips: a way to fail loudly

Here is the pattern that separates the agents that work from the ones that demo, and it's the one that gets cut first under deadline.

Every agentic loop needs a way to check its own work. Without one, errors compound silently. A small mistake on step three becomes the foundation for steps four through forty, and the agent, with no way to notice, reports cheerful success at the end. Anyone who has shipped agents has seen the specific, maddening failure: the agent says it's done, and it isn't, and it filled the gap with a plausible stub.

The fixes are unglamorous and they work. Deterministic gates: a test suite, a type check, a broken-link check, run automatically, that the agent has to pass before it claims victory. A second model reviewing the first, ideally a different one, since two models tend to fail in different ways, so the reviewer catches what the worker faked. The general principle is that an agent verifying its own output is under-tested by definition. Give the loop an external way to fail, and fail loudly, or you are simply trusting a system whose most consistent skill is sounding confident.

This is also why the cheaper-than-you-think setups perform well: a fast, inexpensive model doing the execution, paired with a stronger, pricier one doing the review. The review pass is where the quality comes from, and you can afford to run it because the executor is cheap.

The shape of it

Step back and the patterns rhyme. Workflow before agent. Few tools, well described. A plan on paper instead of everything in the window. The topology that matches the task instead of the most impressive one. A hard validation gate around the whole thing.

None of it is about the model. The model is the part you rent, and it's improving on its own. What you actually design, the part that decides whether your agent works in production or just on stage, is the harness around it: the scope, the tools, the memory, the orchestration, the checks. That's the engineering, and it's where the real product lives.

I build my own agent workflows, and the lesson has been consistent and a little humbling. The versions that survive are never the ambitious ones. They're the boring ones: tightly scoped, sparingly toolled, forced to write down their plan, and not allowed to call themselves done until something other than themselves agrees. The exciting demo is easy. The agent that works on a Tuesday, on a task you didn't anticipate, is just restraint, applied carefully, in five or six unglamorous places.