AI Agent Orchestration: Loops, DAGs and Durable Graphs

The fastest way to overbuild an AI system is to begin with the question, “How many agents should we use?”

Begin somewhere else:

What must remain true while the system is uncertain, stateful, expensive, or partially failing?

That question selects the orchestration pattern. Agent count is a consequence.

This guide covers ten patterns from deterministic pipelines to dynamic graphs. It is the evergreen companion to Graph Engineering: The AI Architect’s Guide, where I define the execution, context, and control planes in detail.

For the worker-level view, start with Six Primitives of Every AI Agent. For runtime and trust-boundary decisions, use AI Architecture 2026. Once the graph is live, the companion is the multi-agent observability stack.

The decision in one table

Five axes that choose your topology

1. Structural uncertainty

Can you know the steps before execution?

• Low uncertainty favors deterministic code and DAGs.
• Medium uncertainty favors routers and bounded loops.
• High uncertainty may justify an orchestrator that creates work dynamically.

2. State duration

Does state live for one model call, one session, one task, or months?

The longer state must survive, the less acceptable an in-memory transcript becomes. Durable workflow history, external artifacts, and versioned schemas move from conveniences to requirements.

3. Parallel separability

Can workers operate independently without repeatedly sharing a mutable artifact?

Anthropic’s multi-agent research system benefited from broad parallel research. The same article warns that tightly dependent tasks—including many coding tasks—offer less parallelism. If workers collide on the same file or decision, coordination can cost more than it saves.

4. Consequence

What happens if the system is wrong?

A brainstorm can tolerate model-owned routing. A payment, deployment, outbound message, legal claim, or data deletion should cross deterministic policy and often human approval.

5. Recovery granularity

When one step fails, must you rerun the whole process, retry one node, compensate for an external side effect, or resume from a checkpoint?

Recovery is where a pleasant demo becomes architecture.

Pattern 0: deterministic pipeline

input → validate → transform → store → notify

Use a pipeline when the steps and order are known. The model may enrich one step, but code owns the route.

Best for: ingestion, document conversion, data cleanup, publishing mechanics, scheduled reports.

Strength: cheap, testable, observable, easy to replay.

Failure mode: teams insert an autonomous planner where a function would be more reliable.

Graduation signal: a step cannot be specified in advance and requires bounded exploration.

Pattern 1: single agent loop

context → decide → tool → observe → verify ↺

This is the basic unit of agency. One model maintains local ownership while it gathers context and acts.

Best for: repository exploration, data analysis, troubleshooting, research with a narrow objective, interactive work.

Required controls: turn limit, time/cost budget, tool permissions, stop conditions, and a definition of done.

Failure mode: the loop becomes a memory substitute. It keeps re-reading, re-planning, or repeating side effects because progress was never serialized.

Graduation signal: the task contains distinct contracts or independent work that should not share one context window.

Pattern 2: router

request → classify → specialist A | specialist B | deterministic path

A router chooses among well-defined paths. Anthropic identifies routing as a foundational workflow pattern; OpenAI’s orchestration primitives can implement it with code, handoffs, or agents-as-tools.

Best for: support categories, risk tiers, modality selection, model routing, tool domains.

Contract: the router returns a typed route and confidence—not a paragraph describing its mood.

Failure mode: overlapping specialists make routing arbitrary. The system adds latency without creating real separation.

Graduation signal: one request needs several paths concurrently, not one selected path.

Pattern 3: fan-out and merge

                         ┌→ worker A ┐
goal → work packets ─────┼→ worker B ┼→ evidence merge → result
                         └→ worker C ┘

Fan-out is the pattern behind research swarms, parallel review, map-reduce, and candidate generation.

Best for: independent source research, separate files or modules, market comparisons, test generation, diverse hypotheses.

Work-packet contract: exclusive scope, expected artifact, evidence requirement, budget, and no shared mutable ownership.

Merge contract: deduplicate, reconcile contradictions, record provenance, and reject unsupported claims.

Failure mode: the synthesizer treats five confident summaries as five independent facts.

Graduation signal: the workers themselves need dynamic decomposition or cross-worker negotiation.

Pattern 4: manager with agents-as-tools

A manager retains conversation and decision ownership. Specialists act like powerful tools and return bounded results.

OpenAI’s Agents SDK distinguishes this from a handoff: an agent used as a tool does not take over the conversation.

Best for: one coherent user experience with specialist research, legal, financial, design, or coding capabilities.

Strength: centralized policy and synthesis.

Failure mode: the manager becomes a context bottleneck and blindly trusts specialist summaries.

Graduation signal: a specialist needs sustained direct interaction or owns a separate lifecycle.

Pattern 5: handoff

In a handoff, a specialist becomes the active owner.

Best for: triage into a specialized support experience, language or jurisdiction transitions, role-based assistance.

Required contract: what context transfers, what does not, who may return control, and what happens when the target refuses or fails.

Failure mode: ownership ping-pongs between agents, each with partial context and no durable case state.

Graduation signal: the handoff is part of a longer business lifecycle that needs explicit states rather than conversational ownership alone.

Pattern 6: finite state machine

draft → review → approved → scheduled → published
          ↓          ↓
       changes    cancelled

A state machine constrains the system to valid lifecycle transitions. Agents may propose transitions; code enforces them.

Best for: content, orders, onboarding, claims, hiring, approvals, incident response.

Strength: invalid states are representable as errors rather than stories the model tells itself.

Failure mode: hidden state remains in chat, while the database says something else.

Graduation signal: transitions cross external systems, require long waits, or must recover across process restarts.

Pattern 7: durable event-driven workflow

event → durable state → activity → wait/approval → activity → outcome
                        ↘ retry / compensate / resume

This pattern treats agent calls as activities inside a durable business process. Temporal’s OpenAI Agents integration is a strong TypeScript example.

Best for: processes lasting minutes to months, payment or messaging side effects, approvals, retries, distributed services, business SLAs.

Strength: event history, durable timers, resumability, and explicit compensation.

Failure mode: model calls or side effects are placed directly in replay-sensitive workflow code. Activities are not idempotent, so retries duplicate real-world actions.

Graduation signal: the system must dynamically invent subgraphs during execution—but the durable workflow should still own authority and state.

Pattern 8: evaluator–optimizer and adversarial gate

candidate → evaluator → pass
    ↑           ↓ fail + evidence
    └────── repair

Anthropic lists evaluator–optimizer as a core workflow. Claude dynamic workflows add adversarial verification, generate-and-filter, tournaments, and loop-until-done.

Best for: code, claims, plans, designs, compliance-sensitive artifacts, high-value decisions.

Evaluator contract: rubric, observable evidence, threshold, budget, and independence from the creator.

Failure mode: the evaluator rewards fluent form rather than the final state. Or the creator and critic share the same blind spot.

Graduation signal: several evaluators or repair routes need to be selected dynamically based on failure type.

Pattern 9: dynamic agent graph

A dynamic graph creates or revises topology during the run. An orchestrator may decide which workers to spawn, which evidence routes to traverse, or which verification subgraph to activate.

Best for: open-ended investigations, large codebases, incident response, scientific search, evolving plans under incomplete information.

Non-negotiable controls:

• maximum depth, width, turns, cost, and wall time;
• versioned work packets and state;
• explicit join and cancellation semantics;
• deterministic authorization for consequential edges;
• checkpoints before expensive branches;
• traceable parent-child relationships;
• a terminal definition of done;
• outcome-based evaluation that tolerates several valid paths.

Failure mode: topology becomes the product. The system launches workers because it can, not because the work is separable.

DAG, state machine, or dynamic graph?

Many production systems combine them: a durable state machine launches a dynamic research graph, whose workers execute small loops, then a deterministic DAG publishes the verified artifacts.

Static graph versus model-routed graph

Use code for routes when the condition is available as structured state:

const next = invoice.total > approvalLimit
  ? "human-approval"
  : "issue-invoice"

Use a model for routing when meaning must be inferred from ambiguous content. The following TypeScript is conceptual—the method names stand in for the SDK and schema library you select:

const route = await triageAgent.run({
  message,
  allowedRoutes: ["billing", "technical", "safety"],
})

routeSchema.parse(route)

Then let policy authorize the transition. A model-routed edge still needs a typed destination, confidence handling, and fallback.

Shared state without shared confusion

Separate at least four scopes:

1. Run state — current inputs, outputs, retries, budgets, and trace IDs.
2. Artifact state — files, documents, plans, code, reports, and their owners.
3. Business state — account, order, publication, issue, or case lifecycle.
4. Knowledge state — facts, sources, relationships, validity intervals, and provenance.

Do not let every agent write every scope. A graph with unrestricted shared memory is an argument with a database attached.

Human-in-the-loop is a state, not a notification

A production approval node must record:

• the exact proposed action and parameters;
• the evidence available to the reviewer;
• the policy that required review;
• approve, reject, or edit semantics;
• expiry and stale-state behavior;
• the identity of the reviewer;
• how the same serialized run resumes.

OpenAI’s Agents SDK can pause on approval and resume serialized state. Claude Code uses permission prompts and hooks for deterministic lifecycle control, while its dynamic workflows currently have more limited ordinary mid-run interaction. The runtime difference matters.

Evaluating orchestration

Do not require one golden path when several paths can be correct. Score:

Outcome

• Was the terminal state correct?
• Did the artifact meet its acceptance criteria?
• Were side effects correct and authorized?

Evidence

• Are material claims supported?
• Is provenance intact through synthesis?
• Was stale or contradictory context handled?

Control

• Were budgets, permissions, and approvals respected?
• Did the system stop when it should?
• Did retries remain idempotent?

Operations

• Cost and latency by successful path;
• retry and recovery success;
• duplicate work;
• bottlenecks and queue time;
• human intervention rate;
• regression by model, node, skill, and graph version.

A practical selection algorithm

1. Write the desired terminal state.
2. List irreversible side effects.
3. Mark the state that must outlive a context window.
4. Identify truly independent work.
5. Define evidence required at every merge.
6. Choose the smallest pattern that protects those conditions.
7. Add one recovery path before adding one more agent.
8. Measure verified outcomes per unit of coordination.

The enduring architecture

Models will change. Vendor names will change. The orchestration problem is more stable:

• deterministic code where the world is known;
• bounded loops where local uncertainty matters;
• specialists where contracts genuinely differ;
• durable state where time and failure matter;
• evidence where truth matters;
• human authority where consequence matters;
• dynamic graphs only where the problem earns them.

The sophisticated system is not the one with the most nodes.

It is the one that can become more capable without becoming less governable.

Choose the implementation boundary

• Use Claude agent engineering when the work belongs inside Claude Code loops, subagents, teams, hooks, or dynamic workflows.
• Use OpenAI Codex delivery graphs when repository Skills, worktrees, CI evidence, and release controls are the center of the system.
• Use Hermes Agent worker graphs when a durable Kanban plane and named worker lanes fit the operating model.
• Use Grok Build parallel coding graphs when Grok should be a bounded coding node rather than the workflow authority.
• Use Google Antigravity with ADK 2.0 when a Google-native stack needs a precise split between coding loop, graph runtime, and durable process history.

Verification receipt

I mapped every pattern here to a current first-party runtime primitive or an original research result, then checked the taxonomy against state duration, consequence, separability, and recovery. The two short TypeScript fragments are intentionally conceptual; they show the policy boundary and must be adapted to a pinned SDK before execution. This page is a selection framework, not a claim that one topology wins every workload.

FAQ

What is AI agent orchestration?

It is the control of routes, state, tools, handoffs, retries, approvals and termination across one or more agent loops. The model may help choose a route; the system still owns the valid destinations and consequences.

When should I use a single agent loop?

Use one loop for a coherent, reversible task that benefits from shared context and can be verified immediately. Add a budget, tool boundary and stop condition before adding another agent.

When should I move from a loop to a graph?

Move when work branches, parallelizes, crosses trust or authority boundaries, must survive a session, or needs selective recovery. A diagram alone is not a reason.

Should I use a DAG or a state machine?

Use a DAG for fixed, acyclic dependencies. Use a state machine for durable business lifecycles where only certain transitions are valid and repair may cycle. A production system can combine both.

Do multi-agent systems always improve results?

No. Anthropic’s current guidance says multi-agent systems often spend several times the tokens of comparable single-agent implementations. They earn that cost when context isolation, parallel breadth or genuine specialization improves the verified outcome.

Primary sources

• Anthropic: Building effective agents
• Anthropic: Multi-agent research system
• Anthropic: Dynamic workflows in Claude Code
• OpenAI Agents SDK: Orchestration
• OpenAI Agents SDK: Human-in-the-loop
• Temporal: OpenAI Agents SDK integration
• LangGraphJS
• LangGraph Graph API
• GitHub Agentic Workflows
• StateFlow
• LLMCompiler
• Stanford SPRINT
• Stanford AgentFlow
• GraphFlow
• MAST: Why Do Multi-Agent LLM Systems Fail?