Stateless Servers, Stateful Payloads: Sessions vs Continuations, Measured in Rust

One Protocol, Three Places to Keep State

One rmcp v3 server — the SDK was three days old at measurement. Three state policies behind one trait (sticky memory, Redis store, sealed continuation), plus a held gRPC stream as reference (different transport and codec). 74,550 samples, six replicates; kill matrix, time-travel fork, 22 security tests; every raw CSV committed. Part 1 of the Typed Agents series.

Source Code: github.com/zs-dima/sessions-vs-continuations
Raw Data: results/ — every measured round, one row each

TL;DR (For Skimmers)

• State on the wire costs nothing measurable at these sizes — inside ±0.27 ms of a memory lookup on loopback, ±0.10 ms at wide-area RTT. A bound, not a win
• Growth does cost: 250 → 923 bytes of state runs about +120 µs per round by round 10 — three of four control-subtracted contrasts exclude zero, and microbenchmarks predict the magnitude
• The session store makes 9 Redis trips per four-call dialogue: 4.84 ms per round with the store 1 ms away, 1.40 ms on loopback. Quote both or neither
• Sticky memory: the dialogue dies with its replica, and 100/100 abandoned dialogues stay held to process exit
• Legal time travel: an old requestState, re-presented, forked one dialogue into two valid bookings — quoted at 1,388.80 and 804.00. Every protection is a SHOULD; the only MUST — at-most-once — needs server-side state back

MCP Didn't Go Stateless

On July 28, 2026 the Model Context Protocol shipped its largest revision since launch, and the retellings mostly agree: MCP went stateless. That is not quite what happened. MCP stopped having an opinion about state. Protocol-level sessions are gone — no more Mcp-Session-Id, no server-held conversation the transport routes back (SEP-2567) — and every server-initiated request became MRTR, Multi Round-Trip Requests (SEP-2322): a tools/call that needs your input returns an InputRequiredResult carrying inputRequests and an opaque requestState; echo it back with your answers and any replica picks the dialogue up.

What the spec deliberately does not say is what goes inside that envelope. A pointer into replica memory, a key into Redis, or the whole serialized dialogue: all three conform. The choice left the protocol; it is yours now. So I built all three behind one trait and measured them; this repository is the result. (No overnight sensation: the RC was frozen May 21 and spent ten weeks in public.) The short version — sessions didn't disappear, they moved into the payload — turns out to be measurable.

The envelope, in rmcp's sealed form (schematic):

requestState = "rs1." + base64url( expiry ‖ state-json ) + "." + base64url( HMAC tag )

The official pattern docs cover the anatomy; rmcp bounds the loop at DEFAULT_MRTR_MAX_ROUNDS = 10.

A gRPC Engineer's Déjà Vu: Three Places for State

The problem is older than MCP: the server asked the client something mid-operation, and the answer must survive a replica change. That in-flight state has only three homes — replica memory behind affinity, an external store, or the wire; cookie-to-JWT, OAuth continuations, and long-polling walked the same doors. This experiment names them:

Note on arm D, provenance file, verbatim: "Arm D uses a different transport and codec and is outside the controlled comparison (controlled=false)." It serves as a latency floor and a failure-mode contrast; it is not a fourth competitor.

The expected comparison, "MRTR versus gRPC", is a category error: MRTR is a pattern (who holds the dialogue state), gRPC a transport (how bytes travel). They are not even alternatives: MCP has a gRPC transport, contributed by Google. MCP got a gRPC transport — and still replaced server-initiated interactions with continuation-passing at the pattern level.

The SEP made this comparison first: its Rationale rejected bidirectional streams (HTTP/2/3 required, no good without long-lived connections, nothing for fault tolerance). Co-written by Mark D. Roth — a gRPC maintainer and member of the gRPC Steering Committee at Google — and sponsored by Caitie McCaffrey: one of the people who knows held streams best chose continuations. The same release removed SSE resumability (SEP-2575): "A broken response stream loses the in-flight request; clients MUST re-issue it as a new request with a new request ID." If arm D's kill-matrix row below looks like a straw man, note that it is the behaviour the spec chose.

A, B and C share one binary, one protocol, one codec, one script, and one machine. Exactly two things differ: the StatePolicy implementation and, for A, the nginx configuration it cannot work without. The asymmetries that remain (C re-sends a growing state, B pays a store lookup, A needs affinity) are not flaws; they are the comparison itself.

And a parallel for the Rust crowd: rustc faced the same choice for async fn and picked the same door — serialize the continuation into a fixed-size state machine instead of holding CPS closures alive. MRTR is that decision at protocol level: make the continuation portable instead of keeping the connection durable.

"Extremely Expensive," Measured

Why surgery rather than repair? The SEP is blunt: adoption of server-initiated requests had been "very low or blocked" for many remote servers (SEP-2322) — SSE streams and server-side state had left the features mostly on paper. Roots and Sampling, two of MRTR's three request types, were deprecated the same day (SEP-2577); the survivor, elicitation — the one this demo drives — waits on a human, and an answer may come in "days or months, or maybe even never" (SEP-2322). A held connection and a replica's memory, both waiting on human attention, scale on no infrastructure.

The SEP argues the rest in prose, without a single number, and never claims MRTR is faster — a latency loss for C refutes nothing. I turned each claim into a metric; full quotes: SEP-CLAIMS.md.

The verdict: "nine of eleven hold, one is weaker than its wording, and one could not be tested at this scale."

One Server, Three Policies

The whole design fits in one trait (policy/mod.rs):

pub trait StatePolicy: Send + Sync + 'static {
    fn id(&self) -> PolicyId;

    /// Dialogue state → the `requestState` the client echoes.
    fn store(&self, ctx: &StateCtx<'_>, state: &DialogState)
        -> impl Future<Output = Result<String, PolicyError>> + Send;

    /// The echoed string → dialogue state. Untrusted input.
    fn load(&self, ctx: &StateCtx<'_>, echoed: &str)
        -> impl Future<Output = Result<DialogState, PolicyError>> + Send;
}

Policy A stores a handle into this replica's map — note the remove: reading consumes the state (sticky.rs):

async fn store(&self, _ctx: &StateCtx<'_>, state: &DialogState) -> Result<String, PolicyError> {
    let handle = format!("mem_{}", Uuid::new_v4());
    self.dialogues.insert(handle.clone(), state.clone());
    Ok(handle)
}

async fn load(&self, _ctx: &StateCtx<'_>, echoed: &str) -> Result<DialogState, PolicyError> {
    // A miss is `NotFound`, never `BackendDown`: this replica never saw this dialogue.
    self.dialogues.remove(echoed).map(|(_, state)| state).ok_or(PolicyError::NotFound)
}

Policy B swaps the map for Redis: SET EX on store, GET then DEL on load (session_store.rs). One design decision there matters: Ok(None) is NotFound ("your dialogue expired"); Err(_) is BackendDown ("our Redis is down" — a different person on call); collapsing them would collapse two kill-matrix rows.

Policy C has no backend: it seals the state onto the wire and verifies what comes back (continuation.rs):

async fn store(&self, ctx: &StateCtx<'_>, state: &DialogState) -> Result<String, PolicyError> {
    let state = self.prepare(state);
    let sealed = match self.mode {
        SealMode::Plain => Self::seal_plain(&state),
        SealMode::Hmac  => self.seal_hmac(ctx, &state)?,
        SealMode::Aead  => self.seal_aead(ctx, &state)?,
    };
    if sealed.len() > self.max_state_bytes {
        return Err(PolicyError::TooLarge { got: sealed.len(), max: self.max_state_bytes });
    }
    if self.measures.single_use && !state.nonce.is_empty() {
        self.nonces.issue(&state.nonce, self.measures.ttl);
    }
    Ok(sealed)
}

// `load` mirrors the dispatch, then `check_single_use`.

Code lines: sticky 37, session store 78, continuation 315 — plus 31 for the nonce store --single-use needs: C's honest total is 346 — bigger because it carries three seal modes and four switchable measures, not because the pattern is hard. Everything runs against that three-day-old SDK (pinned =3.0.0; three discrepancies I hit along the way went upstream, starting with rust-sdk#1094 and #1095) behind a parity gate: every policy and the reference arm must reach the same booking, quote, and round count. The client is rmcp too; no model is in the loop. Bonus: every POST carries Mcp-Method/Mcp-Name (SEP-2243), so nginx logs per-method latency without parsing JSON bodies.

When things die:

Cells: rounds completed of three, one dialogue each; every scene is a pass/fail test, exiting non-zero if reality disagrees. Arm D's cells are argued from design: the reference arm runs as one container.

Look at the Tampered cell again: key skew is the failure a real stateless deployment walks into. Removing affinity moved shared state into key distribution, where a mismatch is a correct rejection and an outage at once.

The Numbers

Round-2 latency; cells are means of six replicate medians, 100 dialogues each:

A cell is worth ±0.2 ms: six replicates span 0.45 ms (A), 0.36 (B), 0.21 (C) at RTT 0. Read no cell without that spread; B's store sits 1 ms away by design.

The headline is a null result with a bound. C − A at RTT 0 is −0.060 ms, 95% CI [−0.265, +0.144], and the interval straddles zero at every distance: putting the whole dialogue state on the wire is not measurable against an in-process map lookup on this rig, inside ±0.27 ms on loopback and ±0.10 ms at wide-area RTT. The negative means do not make the continuation faster; that is what zero looks like when measured. No per-cell p-values: samples within a cell are serially correlated (lag-1 to +0.69, effective N ≈ 18 of 100); the unit of analysis is the replicate — six per cell, arm order alternating, Student-t intervals. The third of the "Three Things I Got Wrong" below belongs in this paragraph as well: policy A's path includes nginx affinity, and the control meant to price that configuration did not replicate — two passes disagreed by 0.32 ms, more than this bound — so part of A-versus-C may belong to the load balancer rather than the policy.

Policy B's figure is meaningless without its store distance: 4.84 ms per round with Redis 1 ms away, 1.40 ms on loopback; some 70% of that gap is a hop injected on purpose. The gap holds at 4.84–4.87 ms across all four panels and decomposes exactly: divide S3's increases by the injected delay and out come 1 / 3 / 3 / 2 Redis trips across the four calls — nine per dialogue, not twelve (SET on open, GET+DEL+SET mid-dialogue, GET+DEL to close; an artefact of this implementation — a denser store client would make fewer). Two independent delays agree; the sweep asserts the multipliers within 15%.

The strongest new result: growth costs. Over ten rounds the state grows 250 → 923 bytes and round 10 runs measurably slower — twelve of twelve replicate-cells positive, while the constant-key controls show nothing:

Three of the four contrasts exclude zero; the fourth misses by a microsecond. That is roughly 7% of a 1.6 ms round, 0.4% of a 27 ms one; the microbenchmarks independently predict 90–100 µs for the same sizes — two instruments agreeing. The cost is linear in size rather than a threshold: 923 bytes is simply where the ten-round script ends; the curve shows no knee.

The durable price is bytes. A and B send a constant 40- and 41-byte key forever; C reaches 1,007 bytes by round 10 lean, 1,590 verbose — that verbosity is a design decision of this demo; the protocol does not require it. (1,007 here against 923 in the growth run is no contradiction: this measurement's rig writes a longer replica name into the state each round, and RESULTS.md reconciles the two tables byte for byte.) Per-round cost is linear in size; cumulative upload is quadratic in rounds. The lookup:

One consequence follows without any new measurement: swarms multiply the curve — dozens of parallel subagents, each with its own MRTR dialogue, scale the quadratic upload by the swarm's width. A dialogue is sequential by design (requestState resumes its originating request, not parallel ones), so a swarm parallelises across dialogues, never inside one. In return, only the continuation can hand a dialogue over: pass the envelope to another process, cluster, or agent, and it continues — the sticky handle is pinned to its replica, the store key to store access. If the host keeps requestState in the model's context (nothing requires it; some will), a 1,590-byte verbose state is roughly 400 tokens a round — an estimate, host-dependent.

Compare policies within a panel, not across panels — each has its own y scale. D is the reference arm: different transport and codec, controlled=false.

Note on the injector: I audited it separately — 8 KiB of filler costs 0.438 ms with no toxic, 0.375 ms under a bidirectional 25 ms one; no per-chunk amplification. Twelve sanity cells, all PASS.

Environment: rustc 1.97.1, Ubuntu 24.04 (WSL2), i7-8700 / 16 GiB, rmcp =3.0.0, six replicates, 74,550 samples; provenance: env.json. git_dirty: true over-reports: it counted the results/ tree the sweep rewrites plus untracked notes; the preflight verified tracked sources matched git_sha.

Three Things I Got Wrong Before Replication

1. The growth question, three times. Single passes answered it "no" (p = 0.15), "a small yes", and "no — the control moved more". Six replicates and a control that cannot show the effect settled it: +119–130 µs, twelve of twelve positive.
2. L2's load skew. One run said p = 0.021: publishable, and wrong. Five more runs scattered between 0.19 and 0.76 (the claims table has the list) — exactly the yield chance predicts at α = 0.05.
3. The load balancer, still open. I assumed policy A's affinity config was neutral. Two single passes of the pricing control disagreed by 0.32 ms — more than the A-versus-C difference itself. Unreplicated controls bound nothing; part of A-versus-C may be nginx, and this rig cannot say how much.

So what does a builder take away? At these sizes the wire is free and the store is not. My rule of thumb (anchors measured, the line between them extrapolated): a kilobyte of state costs a round ~0.1 ms; 8 KiB, ~0.4 ms; 32 KiB, ~1.5 ms. The line is affine — roughly 50 µs flat plus ~45 µs per KiB — and it prices the whole round (serialization and transfer, not just the 12 µs verify above), with the loop capped at ten rounds anyway. There is one tension to keep in view: the anchors are bulk padding, which exercises transfer only, and the measured growth cost (~120 µs for +0.7 KiB) sits well above what the slope alone would price — a real state pays the codec and the JSON decode on top of transfer, and BENCHMARKS.md rebuilds the 90–100 µs prediction from exactly those parts. Treat the line as a floor for grown dialogue state. What compounds is the cumulative upload, so the thing to budget is bytes; the milliseconds follow. Which you can afford is a deployment question, and no benchmark answers it; the matrix below turns it into a decision table.

requestState Under an Auth Engineer's Loupe

Every claim names its source — Spec 2026-07-28 (shipped; governs), SEP-2322 (the proposal; cited to show what changed), author's inference (in neither), because the two documents disagree. Every behavioural sentence has a test: 22 in the security suite, twenty on C, two pinning A and B. None of this is rmcp lore: the envelope lives in the protocol, and every SDK shipping MRTR — TypeScript's included; C#'s v2.0 walked through it the same week — hands its authors the same choices. Whether yours ships a sealing codec (rmcp's is the opt-in rs1.) or leaves you gluing base64 is the first check.

What changed:

Normative force was weakened: all three measures (principal, a short TTL, an originating-request identifier) are now recommendations a conforming server may skip, and this demo runs as exactly that server — four flags, off by default (--bind-principal, --ttl-secs, --bind-origin, --single-use). To be precise about TTL: immortal state is legal only where the SHOULD was skipped, and "short" has no number — a client cannot know how long its state lives. Size is unspecified in both; this server caps it before verifying. The spec's example still prints "requestState": "AEAD-protected blob" beside a rule requiring integrity alone, while the SDK ships HMAC: the gap an implementer trips over.

Replay is the finding. The three SHOULDs bind the state to a principal, a request, and a clock; an earlier state of your own dialogue, re-presented for the same request inside the TTL, passes all three. The warning box, the section's only MUST, says so (Spec 2026-07-28):

Note that these measures bound the replay window and prevent cross-user and cross-request reuse, but do not by themselves guarantee single-use. Servers for which a given requestState must be consumed at most once (e.g., one-time redemptions) MUST enforce that invariant server-side.

The demo runs it end to end: complete a dialogue, re-present the state issued at its start, answer differently. Two bookings result:

One sealed state, two bookings: all three SHOULD measures pass. (The replayed state is the opening call's; a mid-dialogue rewind is equally legal; the demo does not exercise it.) No stateless cryptography closes this. The one configuration that refuses it keeps a server-side nonce store: a session store returning through the back door. A neighbouring gap: "idempotent" appears in neither document, yet MRTR turns one operation into N requests — an honest client that times out and retries may re-execute a side effect.

The two tests pinning A and B point the opposite way. Both consume state by reading it (the map remove()s, the store DELs), so A and B get at-most-once for free; C buys it back with --single-use. Both ignore the request context: a leaked mem_/sess_ handle is honoured for any authenticated caller — a smaller surface because the state never travels, not because it is protected.

On confidentiality, the conclusion is mine, not the spec's. HMAC satisfies the integrity MUST while the payload stays readable — and HMAC is what the SDK ships. The spec permits any encoding: "Servers are free to encode the state in any format (e.g. base64-encoded JSON, encrypted JWT, serialized binary)". All that guards the payload is "Clients MUST NOT inspect, parse, modify": politeness, not cryptography. The demonstration needs no key and no attack (request_state_security.rs):

#[tokio::test]
async fn hmac_seal_does_not_provide_confidentiality() {
    let server = TestServer::start(TestConfig::default().seal(SealMode::Hmac)).await;
    let state = open_and_get_state(&server, "alice").await;

    let body = state.split('.').nth(1).expect("rs1 body");
    let raw = B64.decode(body).expect("base64 decodes without any key");
    // Eight bytes of big-endian expiry, then the payload as plain JSON.
    let json: serde_json::Value =
        serde_json::from_slice(&raw[8..]).expect("the payload parses as JSON");

    assert_eq!(json["principal"], "alice",
        "the signed state names its principal in the clear");
}

My inference: a conformant server may put user data on the wire in plaintext, and the client must store that state to echo it — so it rests wherever dialogue history rests: disk, telemetry, crash dumps, and the model's context window. rmcp says so itself: "signed, not encrypted… Do not put secrets in it." And AEAD would have cost nothing measurable on this rig (BENCHMARKS.md prices the pair). Confidentiality was available for free.

Mind who carries the envelope. In the cookie-and-JWT world the courier is a browser: dumb and predictable. In MCP the courier is an LLM host, and the spec already treats what comes back as attacker-controlled; "MUST NOT inspect" was written for an obedient client, not one that can be argued with. My inference: once an attacker reaches the host's context, re-presenting an old envelope is a legal request; nothing malfunctions. The fork above is what a successful prompt injection buys, no cryptography broken.

Structurally, requestState is a JWS without kid, without a readable exp, without aud, without jti, and without an RFC. The SAML crowd will recognise RelayState. No kid means rotation either kills every dialogue in flight or forces trial decryption (reported upstream). The counterweight is real: with binding on, a stolen copy is useless; lifetimes are seconds; authorization re-evaluates every round — a held stream, authorized once at connect, cannot.

Note: SEP-2243's body still prints -32001 for HeaderMismatch, and deliberately so — SEPs are kept as historical records, with an appended "Changes since SEP became Final" section telling you to read every -32001 as -32020, which is what the shipped spec and rmcp use. The trap is narrow but real: whoever copies the code out of the table without scrolling.

The Matrix: When to Choose What

Each row's verdict in RESULTS.md is marked measured or argued (one row design, one demonstrated), evidence beside it; the portability row is this article's argued addition. Column D is a reference (different transport and codec, controlled=false) and never shares a row's evidence with A/B/C.

One row of that table has no winning column, and it is worth spelling out: the server can no longer speak first. Push did not die with a storage location; it died with the mechanism, and no state policy brings it back. Outside subscriptions/listen the protocol gives a server no way to initiate anything unprompted. Redis returns your data; it cannot tell you whether a live process is still on the far end. With stream resumability removed (SEP-2575), between rounds you do not know whether anyone is still on the line. And the garbage-collection dilemma the SEP holds against the session store — how long to wait for an answer that may come in "days or months, or maybe even never" (SEP-2322) — does not disappear under policy C; it becomes yours, in TTL form.

Part 2 Teaser: Typed Conversations

requestState is a serialized continuation, and a continuation has a phase. Today that phase is a string inside an opaque blob, checked at runtime, if the server remembers to check. It could be a type:

pub struct AwaitingAnswers;
pub struct Done;

pub struct Session<Phase> {
    booking_ref: String,
    phase: PhantomData<Phase>,
}

impl Session<AwaitingAnswers> {
    pub fn answer(self, booking_ref: &str) -> Session<Done> { /* … */ }
}

impl Session<Done> {
    pub fn result(&self) -> &str { &self.booking_ref } // defined once, only for Done
}

Ask for the result before the dialogue is done — the illegal program does not exist:

Not a screenshot: CI renders the figure from the committed compiler output (make teaser), and trybuild (make ui) fails if the illegal dialogues ever compile. The limitation is also the hook for next time: these types do not survive serialization. On the wire, the phase is a string again. Making it survive is Part 2, Typed Conversations — follow along at dmitrii.app.

Conclusion: Sessions Didn't Disappear — They Moved into the Payload

To be exact, MCP did not forbid server-side state. It stopped managing sessions at the protocol level, stopped requiring storage or affinity for the ephemeral case, and directs persistent tools to Tasks. requestState is a neutral envelope: a sticky handle, a store key, and a sealed continuation all conform. MCP didn't pick a side; it moved the choice into an envelope.

The measurements close on the spec's own text: the kill matrix showed what the store pays, the loupe what the continuation pays, and the warning box where the continuation must call the session back. The document whose goal was removing server state concedes, in its only MUST, the one class of correctness that still requires it.

It also joins 2026's loudest infrastructure argument: durable execution answers agent mortality with a journal — persist every step, replay on death. MCP answered by refusing to hold the task: ephemeral dialogues carry their state; persistent work goes to Tasks.

Key Takeaways

1. The envelope is neutral — all three strategies conform; the spec changed the default and left the law alone.
2. State on the wire is free at these sizes; growth is not — A-versus-C sits below the rig's resolution (±0.27 ms); a continuation grown 3.7× costs about 120 µs a round.
3. Each session keeps its bill — nine trips and BackendDown for the store; affinity and NotFound for the replica; quadratic bytes and a Tampered key skew for the continuation.
4. Security became configuration — principal, TTL, and origin binding are SHOULDs a conformant server may omit; the only MUST, at-most-once, brings the session store back through the back door.

Next Steps

🚀 Try It

• Clone: github.com/zs-dima/sessions-vs-continuations
• make demo — one dialogue per policy, no docker, under a minute
• make compose-up scenes — the failure scenes, each a pass/fail test

📚 Further Reading

• MCP Specification 2026-07-28 and its changelog
• SEP-2322: Multi Round-Trip Requests — the claims measured here
• rmcp — the official Rust SDK · discussion #969
• gRPC as a native transport for MCP — Google Cloud
• Why MCP uses JSON-RPC
• In the repo: METHODOLOGY · RESULTS · SECURITY · SEP-CLAIMS · BENCHMARKS
• The arc that led here: Go → Rust → API-First — one auth service built twice, then its contract generated from a single source

🤝 Let’s Connect

I’m a software engineer building high-performance systems. If you have questions or want to clarify details, feel free to reach out.

Blog · GitHub · LinkedIn · X / Twitter · Email

with ❤️ Dmitrii Zusmanovich