AI architecture field guide
Official reference architectures and working repositories, organised by plane rather than publisher. Each entry shows when its documentation and source links were last checked.
Three jobs, not three vendors
Streaming UI, auth, the request that a person is waiting on
Vercel · Cloudflare · Netlify · any Node host
Workers, queues, MCP services, and anything that outlives a request
Railway · Fly.io · Modal · Cloudflare · ECS · Cloud Run
Model endpoints, governed data, hosted evaluation
Vertex · Bedrock · Azure AI · OpenAI · Anthropic
Keep the web request short. Move durable work to something that outlives it. Use managed AI and data services only where their governance or scale earns the complexity. Which vendor fills each job is the reversible part.
Request-scoped
Streaming UI, authentication, API entry, preview delivery. Anything a person is actively waiting on.
Durable runtime
Workers, queues, databases, MCP services, long jobs. Anything that outlives the request that started it.
Managed service
Model endpoints, governed data, hosted evaluation. Buy these where their governance or scale earns the dependency.
Where a long run lives
An agent loop that runs for eleven minutes is a different deployment problem from a completion that returns in two seconds. Every platform below answers it; they differ in what they hand you and what they leave you operating.
Official source atlas
Filtered by the same seven planes the field guide uses below. Pick the plane you cannot currently name an owner for — that is where the next incident comes from.
A Next.js agent surface with streamed responses, tool results, authentication, persistence, and model routing.
Use it for: Public AI products, internal copilots, and fast product iteration.
Vendor-neutral traces, metrics, and logs that connect frontend requests to model calls, tools, workers, and failures.
Use it for: Cross-cloud systems that need one evidence trail across Vercel, Railway, and GCP.
A YAML matrix of prompts, providers, and test cases, scored by deterministic assertions or model graders and diffed side by side in CI.
Use it for: Deciding whether a prompt or model change helped, before it reaches users.
ADK composes coordinator, specialist, sequential, parallel, loop, and graph workflows in code.
Use it for: Teams standardizing agent roles, tools, sessions, and evaluation on Google Cloud.
Agents, handoffs, guardrails, sessions, human-in-the-loop controls, and tracing composed as a backend service.
Use it for: Python agent services that need explicit handoffs and inspectable traces.
Long-running workers, queues, databases, and service-to-service networking behind a separately deployed web experience.
Use it for: Background research, ingestion, browser jobs, and workloads that should not live inside a request lifecycle.
Retries, timeouts, durable state, signals, and human approvals separated from model calls and web requests.
Use it for: Long-running business processes where silent failure is unacceptable.
A governed boundary for exposing tools and context to agents without coupling every integration to one model provider.
Use it for: Shared enterprise tools, GitHub operations, and replaceable model runtimes.
A Google Cloud reference design that stores embeddings in AlloyDB with pgvector, retrieves context in the application, sends contextualized prompts to an LLM on Agent Platform, and evaluates quality separately.
Use it for: Governed knowledge assistants and data-heavy enterprise workloads.
One OpenAI-compatible endpoint in front of many providers, with virtual keys, budget hooks, load-balanced routing that falls back on failure, and per-request spend written to one ledger.
Use it for: Systems that need to swap or lose a provider without touching every call site.
Deploy lanes
Official starting points for the lanes above. Deploying one buys you a working lane — the seams between planes, and what crosses them, still belong to you.
Vercel
The canonical request-scoped agent surface — streaming UI, auth, persistence, and a model gateway, cloned into your own repository.
v0 by Vercel
Prompt-to-component for the surface in front of your loop. Export the code into your repository; the boundaries stay yours to draw.
Vercel
Maintained starters for the request-scoped lane, filterable by stack and use case.
Railway
Durable-runtime starters — workers, queues, Postgres, and the services that outlive a request.
Fork the maintained implementation, replace its boundaries one at a time, and keep deployment, observability, security, and rollback evidence beside the code.
Definition
AI architecture is the arrangement of the parts that surround a model — how requests reach it, what context it is given, which tools it may call, who approves side effects, and how the whole run is measured. The model is one component. Architecture is the decision about everything else.
This page is about software architecture for systems that call language models. It is not about designing buildings, and it is not about accelerator silicon — two different fields that share the phrase. If you are here to decide how to structure an application that calls a model in production, you are in the right place.
The reason architecture carries so much weight is that model quality is the part you least control and least differentiate on. Everyone can reach a frontier model. What separates a system that survives contact with users from one that does not is the boundaries around it: what the model is allowed to see, what it is allowed to do, and what happens when it is wrong.
Reference architecture
Every production system that calls a model has these planes, whether or not anyone drew them. Naming them is useful because failures are almost never located where they are observed — and because each seam is a boundary where something changes character. Read it from the base up: nothing above works if the plane under it is unreliable.
Stream partial work, and let a human interrupt it or approve it.
Record every model call, tool call, and token as one traceable run.
Decide whether a change made the system better, before users do.
Choose the shape: fixed workflow, one agent loop, or many.
Expose capability with schemas, scopes, and an audit trail.
Put the right tokens in the window, and leave the rest out.
Reach a model, survive it being slow, wrong, or gone.
Tool results need an explicit trust boundary. A search result, a fetched page, or a database row may contain instructions written by someone outside your system. Treat retrieved content as data and enforce action permissions in application code. A fluent proposal does not establish authority to act.
The orchestration decision
Build a fixed workflow when you can name every step before the request arrives. Build an agent loop when the steps are unknown but the task is one coherent piece of work. The test is not how capable the model is — it is whether you can enumerate the path in advance.
| Shape | Use when | Cost you accept | How it fails |
|---|---|---|---|
| Fixed workflow | You can name every step before the request arrives. | No adaptation when the input is not what you planned for. | Stays confident and goes silently wrong once reality drifts from the graph. |
| Single agent loop | The steps are unknown but the task is one coherent piece of work. | Latency and spend grow with the length of the loop. | The loop loses track of its own earlier decisions as the window fills. |
| Parallel sub-agents | Breadth-first gathering, where each lookup is independent of the others. | A merge step you have to design, and results that can contradict. | Duplicated work, and summaries that disagree with no way to adjudicate. |
| Sequential sub-agents | Work that mutates shared state and must not interleave. | Throughput. You gave up the parallelism on purpose. | Slow enough that someone proposes parallelising the writes again. |
Multi-agent designs should be compared on the workload they serve. Start with the heuristic: parallelise reads, serialise writes. Independent gathering can reduce latency, but still needs limits on cost, rate, and inconsistent evidence. Concurrent writes require isolation, conflict detection, and reconciliation. Serialize shared mutations until that contract is tested. This is a design recommendation, not a claim that concurrent writes are impossible.
Protocol layer · verified 20 August 2026
The Model Context Protocol revision dated 2026-07-28 removed protocol-level sessions, theMcp-Session-Idheader, and the initializehandshake. Servers that need cross-call state now mint explicit handles and pass them as ordinary tool arguments.
This matters architecturally rather than cosmetically. A stateless protocol core means an MCP server can sit behind ordinary HTTP infrastructure — load balancers, caches, retries — without session affinity. It also means any design that assumed a connection lifecycle now has to carry that state somewhere explicit.
Removed
Sessions, the initialize/initialized handshake, ping, logging/setLevel, and SSE stream resumability.
Added
server/discover for version and capability negotiation, and subscriptions/listen in place of resource subscriptions.
Required
A resultType field on every result. Results from earlier servers that omit it are treated as complete.
Deprecated
Roots, Sampling, and Logging, under a policy guaranteeing a minimum twelve-month window.
Two additions are easy to miss and worth designing around. Caching became a protocol concern: list and read results now carry ttlMs and cacheScope, and servers are asked to return tools in a deterministic order specifically so client-side and prompt caches stay stable. And tracing became one too: OpenTelemetry trace context propagates through _meta keys, so a tool call can join the same trace as the request that caused it.
Source: MCP specification 2026-07-28, Key Changes. Read in full on 20 August 2026. The previous revision was 2025-11-25.
Security baseline · verified 20 August 2026
The OWASP GenAI LLM Top 10 2026 was published in August 2026. Prompt Injection remains first, with Excessive Agency at third. Address excessive agency through scoped tool permissions, approval boundaries, and monitoring. Prompt instructions alone do not enforce those controls.
Three of these are architectural rather than behavioural, and they are the three worth designing for on day one. Excessive Agency is decided by how narrowly you scope tool permissions. Unbounded Consumption is decided by whether loop limits live in code or in a prompt. Hidden Context Exposure — broadened this year beyond system-prompt leakage — is decided by what you put in the window in the first place.
Source: OWASP GenAI LLM Top 10 2026, with the entry list read from the project repository on 20 August 2026. Note that the GenAI Security Project also publishes a separate list for agentic applications; they are frequently conflated.
Field catalog
Sorted by what an operator sees first, because the observed symptom and the real cause almost never sit in the same plane.
Contested ground
Most architecture writing presents the current moment as more settled than it is. These are live disagreements between people who have shipped real systems, and you should expect to make your own call rather than find a consensus to adopt.
Anthropic has published a multi-agent research architecture where sub-agents run in parallel with isolated context. Cognition has argued the opposite case — that parallel sub-agents make independent decisions which then conflict. Both report real results on real systems. The published performance numbers on each side are not independently reproducible, so we do not repeat them here; the workload distinction in the table above is the part that transfers.
As context windows grow, the argument that retrieval is a workaround gets louder. The counter-argument that has not been answered: you cannot apply access control to a context window. If different users are entitled to different documents, retrieval is not an optimisation, it is the enforcement point.
One camp keeps the agent loop small enough to read in one sitting and treats frameworks as indirection. The other argues that durable execution, checkpointing, and replay are genuinely hard and not worth rebuilding. This one usually resolves on operational maturity rather than taste: if you already run durable workflows, the framework buys less than it costs.
Questions
AI architecture is the arrangement of the parts that surround a model: how requests reach it, what context it is given, which tools it may call, who approves side effects, and how the whole run is measured. The model is one component. Architecture assigns boundaries, ownership, and recovery behavior to the surrounding system.
Start with a fixed workflow when you can name the required steps before the request arrives. Consider an agent loop when the path is unknown but the task is one coherent piece of work. A fixed path makes execution easier to bound and test. Compare latency, total cost, and task outcomes on your workload before choosing.
Parallel agents can help when the work is read-heavy and each unit is independent, such as separate lookups or documents. Measure the quality and latency gain against coordination cost. Parallel writes to shared state require explicit isolation, conflict detection, and reconciliation; serialize them by default until that contract is tested.
The 2026-07-28 revision removed protocol-level sessions, the Mcp-Session-Id header, and the initialize handshake, making MCP stateless. Servers that need cross-call state now mint explicit handles passed as ordinary tool arguments. It also added server/discover, replaced resource subscriptions with subscriptions/listen, required a resultType field on every result, and deprecated Roots, Sampling, and Logging.
The OWASP GenAI LLM Top 10 2026, published in August 2026, ranks Prompt Injection first, Sensitive Information Disclosure second, and Excessive Agency third. Address excessive agency through scoped tool permissions, approval boundaries, and monitoring. Prompt instructions alone do not enforce those controls.
Because retrieval and generation are usually measured together. A generation eval grades the final answer, so it cannot distinguish an answer that was wrong from an answer whose evidence was never retrieved. Measure recall separately. Ingestion and chunking sit upstream of anything the model does, so a failure there reaches you disguised as a generation failure.
Specifications cited on this page were read end to end on 20 August 2026. Where a claim is our position rather than a sourced fact, it says so in the sentence. Where a widely repeated number could not be traced to a primary source, it has been left out rather than repeated.
Reading path
This page is the map. Each of these takes one plane of it and goes down a level.
04 · Orchestration
The agentic engineering master guide
The full curriculum behind the orchestration decision — primitives, skills, and subagent state machines.
03 · Tool surface
Anatomy of a production skill
What a capability has to carry before it is safe to expose to a loop.
03 · Tool surface
The A2A agent card
The other protocol in this layer, and where it stops overlapping with MCP.
01 · Model access
First agent primer
Start here if the loop itself is still unfamiliar. Everything above assumes it.
Applied
Deployable blueprints
The reference designs from the atlas above, worked through to something you can stand up.
Applied
The architect learning path
The sequenced route through this material if you are building the skill rather than one system.
Run it
The architecture review asks four questions about the decisions that are expensive to reverse. Answer them for your own system and you get a report naming which are made, which are still open, and which to close first — or install the same rubric into the coding agent that already has the repository open, and let it grep for the evidence instead of asking you.