The supervisor pattern is the production workhorse for multi-agent systems that need variable routing with central accountability. One supervisor agent sits above a pool of worker agents, receives all incoming tasks, decides which worker handles each, collects results, and returns a unified answer. In 2026 this is the pattern that LangGraph, LangChain, CrewAI, and AutoGen all document as “the right way” for most production use cases. This article covers the architecture, implementation patterns, and the specific tradeoffs that make or break supervisor systems at scale.
What a supervisor agent is
A supervisor agent is an LLM-powered router + coordinator. Its system prompt tells it which worker agents exist and when to invoke each. On every incoming task, it:
- Reads the task and the current state
- Decides which worker(s) to invoke next (or whether to answer directly)
- Dispatches work and waits for results
- Updates state with the results
- Decides what to do next (dispatch again, synthesize, or finish)
Unlike a round-robin or sequential pattern, the supervisor’s decisions are dynamic based on the task’s actual needs. Different tasks follow different worker paths.
Why supervisor beats alternatives for most production use cases
vs. Sequential pipeline: handles variable workflows (not every task needs every step).
vs. Peer-to-peer / Swarm: central accountability. If something goes wrong, you look at the supervisor’s decision log. Peer-to-peer makes “why did we end up at agent C” hard to answer.
vs. Monolithic single agent: specialists do their specific jobs better than generalists. A billing-specific agent trained on billing context outperforms a generalist trying to handle billing.
vs. Debate / critique: cheaper. Debate runs multiple candidates in parallel; supervisor runs one worker at a time. Lower cost per task.
Minimal supervisor in LangGraph
from langgraph.graph import StateGraph, END
from typing import TypedDict, Literal
class State(TypedDict):
task: str
results: dict
next_worker: str
final_answer: str
def supervisor(state: State) -> dict:
"""LLM decides which worker should run next (or FINISH)."""
prompt = f"""Task: {state['task']}
Results so far: {state['results']}
Choose the next worker from: researcher, writer, reviewer, FINISH.
Respond with just the worker name."""
choice = llm.invoke(prompt).content.strip()
return {"next_worker": choice}
def researcher(state): return {"results": {**state["results"], "research": do_research(state["task"])}}
def writer(state): return {"results": {**state["results"], "draft": do_write(state["results"])}}
def reviewer(state): return {"results": {**state["results"], "review": do_review(state["results"])}}
def route(state: State) -> str:
nw = state.get("next_worker", "FINISH")
return nw if nw in ["researcher", "writer", "reviewer"] else "finalize"
def finalize(state): return {"final_answer": state["results"].get("review") or state["results"].get("draft", "No output")}
graph = StateGraph(State)
graph.add_node("supervisor", supervisor)
graph.add_node("researcher", researcher)
graph.add_node("writer", writer)
graph.add_node("reviewer", reviewer)
graph.add_node("finalize", finalize)
graph.set_entry_point("supervisor")
graph.add_conditional_edges("supervisor", route, {
"researcher": "researcher", "writer": "writer", "reviewer": "reviewer", "finalize": "finalize"
})
graph.add_edge("researcher", "supervisor")
graph.add_edge("writer", "supervisor")
graph.add_edge("reviewer", "supervisor")
graph.add_edge("finalize", END)
app = graph.compile()
result = app.invoke({"task": "Write a brief on AI observability tools", "results": {}})Classic shape: supervisor decides which worker to invoke, workers always return to supervisor after completing, supervisor eventually decides to FINISH which routes to a finalize node that produces the output.
System prompt patterns for supervisors
Supervisor behavior is 90% system prompt. Three patterns that make supervisor prompts reliable:
1. Explicit worker catalog
supervisor_prompt = """You coordinate three workers to answer user tasks. Workers: - researcher: use when you need external web search or document lookup - writer: use when you need to turn research into a prose answer - reviewer: use when you need to critique or refine an existing draft Respond with a single worker name OR 'FINISH' when the task is complete. Base your decision on the task and the results so far."""
Explicit catalog eliminates ambiguity. Supervisor cannot accidentally invent a new worker.
2. Decision-rule guardrails
supervisor_prompt += """ Decision rules: - Always start with researcher if the task requires current information - NEVER invoke the same worker twice in a row (prevents loops) - FINISH as soon as the current output is sufficient; don't over-polish"""
Hard rules prevent common failure modes. “Same worker twice in a row” prevents a frequent supervisor pathology.
3. Context-aware routing hints
supervisor_prompt += """ Routing hints: - If results.research is empty AND task needs facts: call researcher - If results.research exists AND results.draft is empty: call writer - If results.draft exists AND results.review is empty: call reviewer - If results.review contains 'APPROVED': call FINISH - If results.review contains revision requests: call writer again"""
Specific state-dependent rules make supervisor decisions predictable. The supervisor becomes near-deterministic when state fully determines the next action.
Scaling to many workers
Supervisor works well with 3-7 workers. Beyond ~10 workers, LLMs start to struggle with the catalog. Mitigations:
- Hierarchical supervision: supervisor-of-supervisors. Top-level supervisor dispatches to mid-level supervisors who dispatch to specialists.
- Worker catalog retrieval: instead of listing all 50 workers in the prompt, retrieve the 5 most relevant to the current task via vector search over worker descriptions.
- Capability-based routing: workers declare capabilities (tags like “web_search”, “code_execution”, “document_analysis”); supervisor matches task to capabilities rather than specific worker names.
Human in the loop
The supervisor is the natural place to inject human review. Before invoking high-stakes workers (send_email, charge_card, execute_sql), pause and ask for human approval:
app = graph.compile(
checkpointer=postgres_checkpointer,
interrupt_before=["send_email_worker", "charge_card_worker", "execute_sql_worker"],
)
# Run until first interrupt
state = app.invoke(initial_task, config=config)
# Human reviews pending action
# User approves. Resume:
app.invoke(None, config=config)Supervisor-based human-in-the-loop is cleaner than other patterns because the gate is always at the same point: “supervisor decided to dispatch worker X, pause before dispatch.”
Common supervisor pathologies and fixes
- Infinite worker invocation: supervisor keeps dispatching to the same worker. Fix: add “never invoke same worker twice in a row” rule + MAX_ITERATIONS cap.
- Finishes too early: supervisor FINISH-es before task is complete. Fix: specify sufficient-completion criteria in the prompt (“only FINISH when results.review contains APPROVED”).
- Never finishes: supervisor keeps polishing. Fix: hard cap on total iterations (10-20) + supervisor prompt rule “FINISH as soon as output is sufficient; do not over-polish.”
- Wrong worker picked: ambiguous worker descriptions. Fix: make worker descriptions crisp and non-overlapping. Test on 10-20 example tasks.
- Supervisor becomes bottleneck: every worker’s output goes through supervisor for next decision = N+1 LLM calls per N worker invocations. Fix: for predictable workflows (same path every time), replace supervisor with explicit sequential edges.
Cost profile at scale
Supervisor cost = (worker LLM calls) + (supervisor LLM calls = N+1 for N workers).
For a 5-worker task:
- Worker calls: 5 * $0.02 = $0.10
- Supervisor calls: 6 * $0.04 (bigger model for routing decisions) = $0.24
- Total: $0.34 per task
Supervisor overhead (~70% of cost here) is significant. Mitigations:
- Use a smaller model for the supervisor if routing decisions are simple (GPT-4o-mini can route if the worker catalog is clear)
- Cache supervisor decisions for identical state (same task + same results = same next worker)
- Replace supervisor with explicit edges when the workflow is predictable
When supervisor pattern is wrong
- Workflow is strictly linear every time (just use sequential, no routing needed)
- Workflow is a strict conversation between peers (use swarm or group chat)
- Quality matters more than cost (use debate with multiple candidates + critic)
- Workflow parallelizes trivially (use supervisor-worker fan-out but without the dispatch-and-wait pattern)
- You have 50+ workers (hierarchical supervision or capability-based routing scales better than flat supervision)
Frequently Asked Questions
Official documentation
Can the supervisor be a human instead of an LLM?
Yes. Replace the supervisor node with a UI that lets a human pick the next worker. Useful during early development when you want to understand which workflow paths matter before encoding routing logic in an LLM prompt.
Does the supervisor need the same model as the workers?
No. Common pattern: Claude Opus 5.5 for supervisor (hardest decisions), Claude Sonnet 5.5 for workers (specific execution). Cost-conscious teams use GPT-4o-mini for the supervisor when routing is simple. Mix and match freely.
How does supervisor compare to CrewAI’s hierarchical process?
Same pattern, different wrapping. CrewAI’s hierarchical mode auto-generates a supervisor from your crew’s manager_llm. LangGraph gives you explicit control over the supervisor node. Pick CrewAI if you want zero-config supervision; LangGraph if you want fine-grained supervision logic.
Can workers escalate back to the supervisor during execution?
Yes. A worker can return a state field like escalate: true that the supervisor sees on its next decision. The supervisor’s routing logic reads this field and routes to a dedicated escalation worker or ends the task. Clean error-handling pattern.
How do I handle worker-to-worker dependencies?
Supervisor dispatches worker A, waits for completion, then dispatches worker B (if B needs A’s output). The supervisor’s prompt + state determine the dependency order. For complex multi-step dependencies, consider a planner-executor pattern instead of pure supervision.
Can the supervisor run multiple workers in parallel?
Yes with care. In LangGraph, the supervisor can dispatch multiple workers in a single step by returning a list of worker names; fan-out executes them concurrently; the supervisor waits for all to complete before deciding the next step. More complex to implement cleanly but useful when workers are independent.
Caren Bautista
Technical Writer at PIES IT Solution
Responsible for crafting clear, well-structured, and beginner-friendly content across the platform. Handles the writing, proofreading, and editorial review of tutorials, guides, and documentation to ensure every article is accurate, readable, and easy to follow.
Expertise: Technical Writing · Content Creation · Documentation · Editorial Writing · JavaScript · TypeScript · Python · Python Errors · HTTP Errors · MS Excel · View all posts by Caren Bautista →