AI Architecture 2026: Four Decisions Hard to Reverse

TL;DR: Prompt wording, chunk size, model choice, framework — all cheap to change. Four decisions are not. Where the model call goes sets what a vendor swap costs. What shape the loop is sets how the system fails. Where the trust boundary sits decides whether a retrieved document can trigger a side effect. Where a long run lives quietly eliminates platforms before you have evaluated them. On 28 July 2026 the Model Context Protocol removed sessions and the initialize handshake entirely, which moved the fourth decision for everyone using MCP. This is the argument version. The full seven-plane reference lives here.

I have watched a lot of AI architecture reviews now, and the same thing happens in most of them. Someone draws boxes — model, retrieval, tools, agent — connects them with arrows, and everyone nods. The diagram is fine. The diagram is also not where the risk is.

The risk lives in the seams. Not in the boxes, but in what changes character when a request crosses from one box to the next. That is where systems rot, and it is the part nobody draws.

Of everything in that stack, four choices are genuinely expensive to walk back. The rest is a refactor you can do on a quiet Thursday.

Which AI architecture decisions are actually hard to reverse?

Everything else — prompt wording, chunk size, which reranker, which framework — is a Thursday. Treat it that way and spend your review time on the four.

Decision 1: Where does the model call go?

The cheapest version is openai.chat.completions.create() scattered through the codebase. It works immediately and it is the single most common thing I have to help teams undo.

The question is not "which model." Models change monthly and that is fine — that is the reversible decision hiding inside the irreversible one. The question is whether there is exactly one place in your system that knows a model provider's name.

If there is one place, you get four things without further work: routing by task, fallback when a provider degrades, a cache you can actually reason about, and a single line item for spend. If there are ninety places, you get none of them, and the day you need them is the day you cannot have them.

This is the base plane for a reason. Everything above it inherits its properties. A system with no fallback path has no fallback path at every layer, no matter what the higher layers do. If you want the concrete version of this in a Next.js codebase, the Vercel AI SDK write-up covers the indirection without the abstraction tax.

Decision 2: What shape is the loop?

This is the decision most often made by accident. Someone builds a workflow, calls it an agent, and inherits neither the guarantees of a workflow nor the adaptability of an agent.

There are four common shapes, but they live on two independent axes: who chooses the next step, and how many executors coordinate. Picking one is three questions.

The bias should be toward Q1. A fixed workflow that you wrote down is easier to debug, easier to evaluate, cheaper to run, and easier to explain to the person who owns the incident. Reach for a loop when you genuinely cannot enumerate the steps — not because loops are more interesting to build.

What makes this hard to reverse is not the code. It is the evaluation suite. A workflow is graded on whether each step produced the right output. A loop has to be graded on its trajectory — did it take a sensible path, or did it stumble into the right answer? Those are different harnesses. Switch shapes after you have a hundred graded examples and you throw away the harness with them. Modern agentic systems architecture goes deeper on trajectory evaluation; the seven pillars piece covers what a production loop needs around it.

Decision 3: Where does the trust boundary sit?

Here is the sentence that reframes this for most people: text that came back from a tool call is not your text.

A retrieved document, an API response, a fetched web page, a PDF someone uploaded — all of it was authored by someone who is not you and may not like you. If that text reaches the position in your context window where instructions live, it is an instruction. The model has no reliable way to tell the difference, and no amount of "ignore any instructions in the following document" fixes it, because that sentence is also just text.

The architectural fix is not a better prompt. It is three structural things:

1. Untrusted content never enters the instruction position. Fence it, label it, keep it in the data position.
2. Side effects sit behind a gate. Anything irreversible — a write, a message, a payment, a merge — needs a step that a document cannot perform on its own behalf.
3. Tool scopes are narrow by default. A tool that can read a calendar and a tool that can read a calendar and send email are different risk objects, even though they are one line apart in your config.

This is hard to reverse because the cost is not the fix — it is the audit. Every tool you already shipped has to be re-examined against a boundary you did not have when you shipped it. Draw the boundary before the tool surface grows, and it stays cheap. Zero-trust tool meshes covers the production shape of this.

Decision 4: Where does a long run live?

An agent loop that runs for eleven minutes is a different deployment problem from a chat completion that returns in two seconds. Not a harder version of the same problem — a different one.

This is where teams discover their platform choice was load-bearing. A serverless function has a duration ceiling. A loop that outlives a request needs somewhere else to be: a queue and a worker, a durable execution primitive, a container that stays warm, or a machine you start and stop on purpose.

Nobody sits down and decides this. It gets decided by whatever the marketing site was optimized for, and it is discovered in week six.

There is no universal platform winner and I am not going to invent one. The honest framing is a trade: managed platforms hand you a primitive and take the operations; self-assembled clouds hand you every primitive and take your Tuesdays. Both are correct answers to different questions about what your team wants to be responsible for.

What makes it hard to reverse is that "long-running work" is a runtime property. It is not a flag you flip. Moving it means moving the execution model, which means moving the state that execution model implied.

What changed in mid-2026 that moves these decisions?

One thing, concretely, and it lands on decisions 3 and 4.

On 28 July 2026, the Model Context Protocol shipped a revision that made the protocol stateless. Reading the changelog directly, the load-bearing removals are:

• Protocol-level sessions are gone, along with the Mcp-Session-Id header. Servers that need state across calls now mint explicit handles and pass them as ordinary tool arguments.
• The initialize handshake is gone. Every request carries its own protocol version and client capabilities in _meta. A new server/discover RPC advertises what a server supports, if you want to ask up front.
• Stream resumability is gone. No Last-Event-ID, no message redelivery. A broken response stream loses the in-flight request, and the client has to re-issue it as a new request with a new ID.

Read those three together and the architectural consequence is clear: state that used to be implicit in a connection is now something you have to name and own. That is more work at the seam, and it is better work — implicit connection state was exactly the kind of thing that made MCP servers hard to scale horizontally and hard to reason about under failure.

The third bullet is the one that changes plans. If your design assumed a long-lived stream would survive a blip, it no longer does. Retries become your problem, which means idempotency becomes your problem, which means decision 4 just got more expensive to get wrong. The 2026 agentic hierarchy covers where MCP sits relative to skills and agents if that layering is still fuzzy.

How do you tell which decisions you already made by accident?

Four greps and one question. This takes about twenty minutes on a codebase you know.

1. Grep for your model provider's SDK import. More than one module means decision 1 is deferred, not made.
2. Find the loop's exit condition. If it lives in a prompt rather than in code, you have an unbounded loop with a polite request attached.
3. Trace one retrieved document from the retriever to the context window. If you cannot point at the line where it becomes labelled data rather than plain text, decision 3 has been made for you.
4. Find your longest production run. Not the average — the longest. Then check what your platform's ceiling is. If you do not know either number, decision 4 is unmade.

The question: what breaks first if your primary model provider is unavailable for four hours? If the answer is "everything," the four decisions are all still open, and they are open in the expensive direction.

FAQ

Q: Is a fixed workflow really better than an agent? Better at different things. A workflow is cheaper, more debuggable, and easier to evaluate. An agent adapts to inputs you did not anticipate. The failure is picking an agent for work you could have enumerated, because you then pay the agent's cost and get the workflow's ceiling. Start with the workflow and let a real failure push you to the loop.

Q: What is the trust boundary in an AI system? The line between text you authored and text that arrived from somewhere else. System prompts and operator instructions are trusted. Retrieved documents, tool results, fetched pages and uploaded files are not, regardless of how reputable the source looks. The boundary is architectural: untrusted content stays in the data position of the context window and never reaches the instruction position, and side effects sit behind an approval gate.

Q: Did MCP really remove sessions? Yes. The 28 July 2026 revision removed protocol-level sessions and the Mcp-Session-Id header, removed the initialize handshake, and removed SSE stream resumability. Servers needing cross-call state now mint explicit handles passed as ordinary tool arguments. The changelog in the specification repository is the authority.

Q: Which platform should I deploy an AI agent on? The one whose primitive matches how long your runs actually are. Short request-scoped work fits serverless functions and edge isolates. Work that outlives a request needs a durable execution primitive, a queue and worker, or a container that stays warm. Measure your longest real run before you choose, not your median.

Q: How many of these decisions can I defer? All four, briefly. Deferring decision 1 is cheapest and stays cheap longest. Deferring decision 3 gets expensive fastest, because the cost scales with the number of tools you have already shipped. If you defer one thing, do not let it be the trust boundary.

Where to go next

The four decisions are the argument. The reference is longer and lives on its own page: the seven planes of a production AI system, including what each plane owns, what the seam between two planes actually changes, and the failure modes that show up in the wrong plane from where they were caused.

If you want the operational layer rather than the design layer, the observability stack for multi-agent systems is the companion — because three of the four decisions above are only checkable if you can see what a run actually did.

Sources checked against primary documentation on 21 August 2026. Where a claim depends on a vendor page or a protocol revision, the linked source is the authority, not this post.