Multi-Agent Systems
What Is a Multi-Agent System?
A multi-agent system (in the LLM sense) is a setup where several language-model “agents” — each its own prompt, role, and often its own tools — work together to solve one task. Instead of asking a single model to do everything in one giant context window, you split the job across specialists that talk to each other: a researcher gathers facts, a writer drafts, a reviewer critiques, and a coordinator stitches the pieces together.
The intuition is the same one behind a human team. You could write a report alone, or you could hand research, drafting, and editing to different people. A team can parallelize and bring focused expertise — but it also adds meetings, handoffs, and the chance that one person's mistake propagates. Multi-agent systems inherit both sides of that trade.
Not the same as multi-agent RL. This lesson is about orchestrating LLM agents that collaborate through natural-language messages. Multi-agent reinforcement learning — where policies are trained to compete or cooperate in an environment — is a separate topic with its own lesson.
Common Topologies
How the agents are wired together — who talks to whom — is the system's topology. A handful of patterns cover most real systems:
Supervisor / orchestrator–workers
A manager agent decomposes the task into subtasks, delegates each to a worker, then synthesizes the results into a final answer. Workers can run in parallel and never talk to each other directly — all coordination flows through the supervisor. This is the most common production pattern.
Pipeline / sequential
Agents are arranged in a chain: the output of one becomes the input of the next (research → draft → edit). Simple and predictable, with no central coordinator — but strictly sequential, so there is no parallelism and an early error flows downstream.
Hierarchical
Supervisors of supervisors. A top-level manager delegates to mid-level managers, each of which owns a team of workers. Useful when a task has nested structure (e.g. a “research lead” coordinating several researchers), but coordination cost grows with depth.
Debate / critic
Two or more agents argue or critique each other's answers, optionally with a judge agent picking a winner or a critic looping until the answer passes. Good for adversarial review and reducing confident-but-wrong outputs — at the cost of extra rounds.
These are building blocks, not mutually exclusive: a real system might use an orchestrator–workers core where one “worker” is itself a critic loop.
Role Specialization
Each agent is usually just an LLM given a focused system prompt, a narrow set of tools, and a clear job. Specialization helps because a tight prompt (“you are a meticulous code reviewer; only report real bugs”) keeps the model on-task far better than one mega-prompt that asks a single agent to research, write, and critique at once. Common roles:
- Planner / orchestrator: decomposes the goal, decides who does what, and merges results. Owns the “big picture” and rarely touches tools directly.
- Researcher: gathers information — searches, reads documents, queries APIs — and returns digested findings.
- Coder / writer: produces the artifact (code, prose, a query, a plan) from the gathered material.
- Reviewer / critic: checks the artifact for correctness, style, or safety, and returns concrete fixes — the adversarial-review role.
Separation of concerns is the real payoff. A reviewer with fresh eyes and no stake in the draft catches mistakes the writer is blind to — much like asking a colleague to proofread rather than re-reading your own work.
Interactive Demo: Orchestrator–Workers Visualizer
Pick a topology and a task, then step through the message flow. Watch the supervisor decompose the task and dispatch subtasks (solid arrows) to the researcher, writer, and reviewer, then results flow back (dashed arrows) and the supervisor synthesizes. Compare the cost readouts across topologies to feel the parallelism benefit and the coordination overhead.
Try this: switch between Single agent and Orchestrator–workers on the same task. The orchestrator spends extra calls on planning and synthesis (higher total cost), but because the workers run in parallel its critical path is the longest single worker, not the sum — that is the latency win. The pipeline trades the parallelism away for simpler, sequential handoffs. Numbers are illustrative; worker outputs are mocked and deterministic.
How Agents Communicate
Agents do not share a brain — they share text. The communication mechanism is a core design choice:
- Message passing: one agent sends a structured message (a task, a result) to another, typically routed through the orchestrator. Clean and auditable, but everything the receiver needs must be in the message.
- Shared scratchpad / blackboard: a common workspace (a document, a memory store) that all agents read from and write to. Avoids re-sending context, but agents can step on each other and the scratchpad can balloon past the context window.
- Handoffs: one agent transfers control and the conversation to another (“this is a billing question — handing off to the billing agent”). The receiver continues the same thread. Common in customer-support routing.
Context is not free. Every message is tokens. A naïve design that forwards the entire transcript to every agent multiplies cost fast. Good systems pass summaries, not raw history.
When Multi-Agent Beats a Single Agent — and When It Does Not
Multi-agent is a tool, not a default. It pays off when the structure of the task matches the structure of the team; it backfires when you add coordination for its own sake.
| Multi-agent helps when… | Multi-agent hurts when… |
|---|---|
| Subtasks are independent and parallelizable (research 5 topics at once) — wall-clock latency drops. | The task is small or inherently sequential — you pay coordination cost for no parallelism. |
| Separation of concerns matters — distinct skills/tools/prompts that would conflict in one agent. | Error propagation dominates — one bad subtask result silently corrupts everything downstream. |
| You want adversarial review — an independent critic catches confident-but-wrong answers. | Added latency & cost from extra planning/synthesis calls is not worth the quality gain. |
| The task is large enough that a single context window would overflow or lose focus. | Coordination overhead (routing, message bloat, debugging across agents) exceeds the benefit. |
Default to the simplest thing that works. A well-prompted single agent with good tools beats a sprawling multi-agent system for most tasks. Reach for multiple agents when you can name a concrete reason — parallelism, separation of concerns, or independent review — not because it sounds sophisticated.
Rule of thumb on cost. Multi-agent almost always uses more tokens than a single agent (planning + synthesis + redundant context). The justification is faster wall-clock time (parallelism) or better quality (review) — not cheaper runs.
Frameworks (High Level)
You can build all of this with raw API calls and a loop, but several frameworks provide the plumbing — message routing, role definitions, and state:
- AutoGen: conversational agents that message each other, including group-chat and supervisor patterns; strong support for code-executing agents and human-in- the-loop.
- CrewAI: a higher-level, role-first abstraction — you declare agents with roles and goals plus tasks, and a “crew” runs them sequentially or hierarchically.
- LangGraph: models the system as an explicit graph of nodes and edges with shared state; lower-level and flexible, good for custom topologies, cycles (e.g. critic loops), and fine control over routing.
They differ mostly in abstraction level: CrewAI is opinionated and quick to start, LangGraph is explicit and controllable, AutoGen sits in between with a conversation-centric model. None of them removes the core trade-offs above — they just make the wiring less tedious.
Key Takeaways
- A multi-agent system orchestrates several specialized LLM agents that collaborate over natural-language messages — distinct from multi-agent reinforcement learning.
- Core topologies: supervisor/orchestrator–workers (delegate & synthesize), pipeline (sequential handoffs), hierarchical (managers of managers), and debate/critic (adversarial review).
- Specialize roles — planner, researcher, coder, reviewer — because a focused prompt and a fresh-eyes critic outperform one mega-prompt.
- Agents communicate via message passing, a shared scratchpad/blackboard, or handoffs; remember that every message costs tokens, so pass summaries, not full transcripts.
- Multi-agent wins on parallelizable subtasks, separation of concerns, and adversarial review; it loses to latency/cost, error propagation, and coordination overhead when the task does not need it.
- It almost always costs more tokens than a single agent — the payoff is speed (parallelism) or quality (review), so default to the simplest design that works.
- Frameworks like AutoGen, CrewAI, and LangGraph provide the plumbing but not the judgment about when to use it.