AI architecture field guide
Official reference architectures and working repositories, organised by the plane they belong to rather than the vendor that published them. Every external link in this catalog was checked on 12 July 2026.
Three jobs, not three vendors
Streaming UI, auth, the request that a person is waiting on
Vercel · Cloudflare · Netlify · any Node host
Workers, queues, MCP services, and anything that outlives a request
Railway · Fly.io · Modal · Cloudflare · ECS · Cloud Run
Model endpoints, governed data, hosted evaluation
Vertex · Bedrock · Azure AI · OpenAI · Anthropic
Keep the web request short. Move durable work to something that outlives it. Use managed AI and data services only where their governance or scale earns the complexity. Which vendor fills each job is the reversible part.
Request-scoped
Streaming UI, authentication, API entry, preview delivery. Anything a person is actively waiting on.
Durable runtime
Workers, queues, databases, MCP services, long jobs. Anything that outlives the request that started it.
Managed service
Model endpoints, governed data, hosted evaluation. Buy these where their governance or scale earns the dependency.
Where a long run lives
An agent loop that runs for eleven minutes is a different deployment problem from a completion that returns in two seconds. Every platform below answers it; they differ in what they hand you and what they leave you operating.
Official source atlas
A Next.js agent surface with streamed responses, tool results, authentication, persistence, and model routing.
Use it for: Public AI products, internal copilots, and fast product iteration.
A grounded retrieval path built around Vertex AI, managed data services, evaluation, and enterprise controls.
Use it for: Governed knowledge assistants and data-heavy enterprise workloads.
Specialized agents coordinated through Google's Agent Development Kit with deployable samples and managed runtime options.
Use it for: Teams standardizing agent roles, tools, sessions, and evaluation on Google Cloud.
Long-running workers, queues, databases, and service-to-service networking behind a separately deployed web experience.
Use it for: Background research, ingestion, browser jobs, and workloads that should not live inside a request lifecycle.
Agents, handoffs, guardrails, sessions, human-in-the-loop controls, and tracing composed as a backend service.
Use it for: Python agent services that need explicit handoffs and inspectable traces.
A governed boundary for exposing tools and context to agents without coupling every integration to one model provider.
Use it for: Shared enterprise tools, GitHub operations, and replaceable model runtimes.
Retries, timeouts, durable state, signals, and human approvals separated from model calls and web requests.
Use it for: Long-running business processes where silent failure is unacceptable.
Vendor-neutral traces, metrics, and logs that connect frontend requests to model calls, tools, workers, and failures.
Use it for: Cross-cloud systems that need one evidence trail across Vercel, Railway, and GCP.
Deploy lanes
Official starting points for the lanes above. Deploying one buys you a working lane — the seams between planes, and what crosses them, still belong to you.
Vercel
The canonical request-scoped agent surface — streaming UI, auth, persistence, and a model gateway, cloned into your own repository.
v0 by Vercel
Prompt-to-component for the surface in front of your loop. Export the code into your repository; the boundaries stay yours to draw.
Vercel
Maintained starters for the request-scoped lane, filterable by stack and use case.
Railway
Durable-runtime starters — workers, queues, Postgres, and the services that outlive a request.
Fork the maintained implementation, replace its boundaries one at a time, and keep deployment, observability, security, and rollback evidence beside the code.
Definition
AI architecture is the arrangement of the parts that surround a model — how requests reach it, what context it is given, which tools it may call, who approves side effects, and how the whole run is measured. The model is one component. Architecture is the decision about everything else.
This page is about software architecture for systems that call language models. It is not about designing buildings, and it is not about accelerator silicon — two different fields that share the phrase. If you are here to decide how to structure an application that calls a model in production, you are in the right place.
The reason architecture carries so much weight is that model quality is the part you least control and least differentiate on. Everyone can reach a frontier model. What separates a system that survives contact with users from one that does not is the boundaries around it: what the model is allowed to see, what it is allowed to do, and what happens when it is wrong.
Reference architecture
Every production system that calls a model has these planes, whether or not anyone drew them. Naming them is useful because failures are almost never located where they are observed — and because each seam is a boundary where something changes character. Read it from the base up: nothing above works if the plane under it is unreliable.
Stream partial work, and let a human interrupt it or approve it.
Record every model call, tool call, and token as one traceable run.
Decide whether a change made the system better, before users do.
Choose the shape: fixed workflow, one agent loop, or many.
Expose capability with schemas, scopes, and an audit trail.
Put the right tokens in the window, and leave the rest out.
Reach a model, survive it being slow, wrong, or gone.
The trust boundary at the tool surface is the one most systems get wrong. Everything a tool returns — a search result, a fetched page, a database row someone else wrote — is input from outside your system. Treating it as instruction rather than data is the single most common way an agent ends up doing something nobody asked for.
The orchestration decision
Build a fixed workflow when you can name every step before the request arrives. Build an agent loop when the steps are unknown but the task is one coherent piece of work. The test is not how capable the model is — it is whether you can enumerate the path in advance.
| Shape | Use when | Cost you accept | How it fails |
|---|---|---|---|
| Fixed workflow | You can name every step before the request arrives. | No adaptation when the input is not what you planned for. | Stays confident and goes silently wrong once reality drifts from the graph. |
| Single agent loop | The steps are unknown but the task is one coherent piece of work. | Latency and spend grow with the length of the loop. | The loop loses track of its own earlier decisions as the window fills. |
| Parallel sub-agents | Breadth-first gathering, where each lookup is independent of the others. | A merge step you have to design, and results that can contradict. | Duplicated work, and summaries that disagree with no way to adjudicate. |
| Sequential sub-agents | Work that mutates shared state and must not interleave. | Throughput. You gave up the parallelism on purpose. | Slow enough that someone proposes parallelising the writes again. |
The multi-agent question is where practitioners most visibly disagree, and the disagreement is usually reported as a contradiction when it is really a difference in workload. The heuristic that reconciles the two camps: parallelise reads, serialise writes. Independent gathering parallelises cleanly because nothing the sub-agents do can conflict. Shared mutation does not, because two agents editing the same state produce lost updates that no merge step can adjudicate after the fact. That is our position, not a citation.
Protocol layer · verified 20 August 2026
The Model Context Protocol revision dated 2026-07-28 removed protocol-level sessions, theMcp-Session-Idheader, and the initializehandshake. Servers that need cross-call state now mint explicit handles and pass them as ordinary tool arguments.
This matters architecturally rather than cosmetically. A stateless protocol core means an MCP server can sit behind ordinary HTTP infrastructure — load balancers, caches, retries — without session affinity. It also means any design that assumed a connection lifecycle now has to carry that state somewhere explicit.
Removed
Sessions, the initialize/initialized handshake, ping, logging/setLevel, and SSE stream resumability.
Added
server/discover for version and capability negotiation, and subscriptions/listen in place of resource subscriptions.
Required
A resultType field on every result. Results from earlier servers that omit it are treated as complete.
Deprecated
Roots, Sampling, and Logging, under a policy guaranteeing a minimum twelve-month window.
Two additions are easy to miss and worth designing around. Caching became a protocol concern: list and read results now carry ttlMs and cacheScope, and servers are asked to return tools in a deterministic order specifically so client-side and prompt caches stay stable. And tracing became one too: OpenTelemetry trace context propagates through _meta keys, so a tool call can join the same trace as the request that caused it.
Source: MCP specification 2026-07-28, Key Changes. Read in full on 20 August 2026. The previous revision was 2025-11-25.
Security baseline · verified 20 August 2026
The OWASP GenAI LLM Top 10 2026 was published on 4 August 2026. Prompt Injection remains first. The move architects should read is Excessive Agency at third — a design fault, not a model fault, and one that is fixed by scoping permissions rather than by prompting more carefully.
Three of these are architectural rather than behavioural, and they are the three worth designing for on day one. Excessive Agency is decided by how narrowly you scope tool permissions. Unbounded Consumption is decided by whether loop limits live in code or in a prompt. Hidden Context Exposure — broadened this year beyond system-prompt leakage — is decided by what you put in the window in the first place.
Source: OWASP GenAI LLM Top 10 2026, with the entry list read from the project repository on 20 August 2026. Note that the GenAI Security Project also publishes a separate list for agentic applications; they are frequently conflated.
Field catalog
Sorted by what an operator sees first, because the observed symptom and the real cause almost never sit in the same plane.
Contested ground
Most architecture writing presents the current moment as more settled than it is. These are live disagreements between people who have shipped real systems, and you should expect to make your own call rather than find a consensus to adopt.
Anthropic has published a multi-agent research architecture where sub-agents run in parallel with isolated context. Cognition has argued the opposite case — that parallel sub-agents make independent decisions which then conflict. Both report real results on real systems. The published performance numbers on each side are not independently reproducible, so we do not repeat them here; the workload distinction in the table above is the part that transfers.
As context windows grow, the argument that retrieval is a workaround gets louder. The counter-argument that has not been answered: you cannot apply access control to a context window. If different users are entitled to different documents, retrieval is not an optimisation, it is the enforcement point.
One camp keeps the agent loop small enough to read in one sitting and treats frameworks as indirection. The other argues that durable execution, checkpointing, and replay are genuinely hard and not worth rebuilding. This one usually resolves on operational maturity rather than taste: if you already run durable workflows, the framework buys less than it costs.
Questions
AI architecture is the arrangement of the parts that surround a model — how requests reach it, what context it is given, which tools it may call, who approves side effects, and how the whole run is measured. The model is one component. Architecture is the decision about everything else, and it is where almost all production failure lives.
Build a fixed workflow when you can name every step before the request arrives. Build an agent loop when the steps are unknown but the task is one coherent piece of work. The test is not how capable the model is; it is whether you can enumerate the path in advance. If you can, the workflow will be cheaper, faster, and easier to debug.
When the work is read-heavy and each unit is genuinely independent — separate lookups, separate documents, separate sources — parallel sub-agents earn their complexity. When the work mutates shared state, they do not: two agents writing the same thing produce lost updates and contradictions no merge step can adjudicate. Parallelise reads, serialise writes.
The 2026-07-28 revision removed protocol-level sessions, the Mcp-Session-Id header, and the initialize handshake, making MCP stateless. Servers that need cross-call state now mint explicit handles passed as ordinary tool arguments. It also added server/discover, replaced resource subscriptions with subscriptions/listen, required a resultType field on every result, and deprecated Roots, Sampling, and Logging.
The OWASP GenAI LLM Top 10 2026, published 4 August 2026, ranks Prompt Injection first, Sensitive Information Disclosure second, and Excessive Agency third. Excessive Agency rising to third is the notable move for architects: it is a design fault, not a model fault, and it is fixed by scoping tool permissions rather than by prompting more carefully.
Because retrieval and generation are usually measured together. A generation eval grades the final answer, so it cannot distinguish an answer that was wrong from an answer whose evidence was never retrieved. Measure recall separately. Ingestion and chunking sit upstream of anything the model does, so a failure there reaches you disguised as a generation failure.
Specifications cited on this page were read end to end on 20 August 2026. Where a claim is our position rather than a sourced fact, it says so in the sentence. Where a widely repeated number could not be traced to a primary source, it has been left out rather than repeated.
Reading path
This page is the map. Each of these takes one plane of it and goes down a level.
04 · Orchestration
The agentic engineering master guide
The full curriculum behind the orchestration decision — primitives, skills, and subagent state machines.
03 · Tool surface
Anatomy of a production skill
What a capability has to carry before it is safe to expose to a loop.
03 · Tool surface
The A2A agent card
The other protocol in this layer, and where it stops overlapping with MCP.
01 · Model access
First agent primer
Start here if the loop itself is still unfamiliar. Everything above assumes it.
Applied
Deployable blueprints
The reference designs from the atlas above, worked through to something you can stand up.
Applied
The architect learning path
The sequenced route through this material if you are building the skill rather than one system.