Read path

A read never declares whether it wants hot or cold data; its position does. Every page a star serves is classified by where the requested sequence number sits relative to the committed ref frontier, and that decision picks the tier: the leader's in-memory tail window above the frontier, or object storage below it, served by any star. This page follows one read through both tiers and the machinery around them.

Position picks the tier

A stream's tail window is the span above the ref frontier: records that are acknowledged and durable but not yet covered by a committed ref. It is small by construction: the stream's slice of the leader's backlog, bounded by the backlog budget. Reads inside it are hot and leader-served; everything below is cold and served by any star from refs. A page never spans tiers: one that starts below the frontier returns short at it, and the next request classifies fresh.

The order matters. The trim watermark is checked first. The tail window cache is consulted before any ref lookup, because a hit is always safe: the retained records are immutable and either acknowledged or already ref-committed. Only then does the reader ask the metastore whether a ref covers the position. While a star leads the stream none of this costs a metastore read: the trim watermark lives in leader memory (trim is holder-served), and only the cold path pays the frontier lookup.

The leader serves the tail window

Above the frontier only the leader can answer. Its flight index knows exactly which records are acknowledged, and the empty answer ("caught up") is a claim only the leader's own acked cursor can make. A star that does not hold the stream answers NOT_OWNER with the holder's address, and the cluster client redials there once, exactly as it does for an append. Nova never forwards a read internally.

The bytes come from the tail window cache (--tail-window-cache-mib, default 128 MiB, one budget shared across both classes). At acknowledgement the leader retains each flight's per-stream payload slices (the pool's own buffers, never a copy), and direct indexing on the contiguous-ID invariant serves a hot read, with no footer involved. The cache is memory-first: when the absorb retires a flight, its payloads stay resident until the byte budget evicts them, oldest-admitted first, so a follower up to budget ÷ ingest behind the absorb is still served from memory after its records have turned cold. The cache is never authoritative: a miss falls back to ranged GETs into the transit bundle, and a flight larger than the whole budget is skipped, logged, and counted.

Subscriptions park on the wake

A subscription on the leader does not poll. It captures the stream's wake (a channel the leader closes at every acknowledgement that moves the frontier and at the leadership's departure) before its read, and parks on it after an empty one, so no edge is lost. The wake carries nothing; the read that follows is the only authority, and a spurious wake is harmless. A parked subscription costs the metastore nothing while nothing happens.

Anywhere else a subscription polls, backing off from 5 ms to a 250 ms cap and re-checking the lease on each empty page. A foreign hot window always redirects, for the subscription's whole life: a holder elsewhere ends it with NOT_OWNER and the SDK reopens on the holder at its cursor; a leadership departing under a parked subscription wakes it and ends it the same way, so the SDK reopens wherever placement puts the stream next, or at the tombstone when the departure was a deletion. A departure that cannot yet certify coverage parks before the wake closes; followers stay, correctly, since the lease is still held there and no successor can append.

Vacant streams

A vacant stream holds no writer lease. With no writer nothing can be in flight, so committed state is everything: any star serves it from refs, and a subscription polls at the 250 ms cadence, redirecting the moment a lease appears. What a vacant stream cannot do is serve a hot read from a live flight index. If a leader died with acknowledged records above the frontier, those records stay invisible until settlement (the successor's fence, or the settle duty for a star that never returns) commits refs over the resident span. A read that meets that residue converges through settlement rather than guessing, bounded by the caller's own deadline. See settlement.

Cold reads: refs, then ranged GETs

Below the frontier the reader resolves the ref covering the position with one floor lookup in the refs keyspace. Two kinds share it:

• A span ref, written by the absorb, covers a stream's records across the chain slots of one absorb batch; its per-slot index (slot, first ID, footer window) locates the transit bundle and the footer to read.
• A catalog row, written by the settle, covers a contiguous run of settled objects; its packed boundary list derives every object key and footer window.

From either, the read has one shape: a ranged GET of the bundle's footer (the manifest, whose sorted fixed-width chunk index lets a point question decode one directory entry instead of the whole footer), then ranged GETs of exactly the chunks the page needs. Never a whole-object fetch. The reader pipelines the walk with up to 8 refs' footers and extents in flight while the current extent streams. Express-class transit bundles live in the express bucket set; a star's reads prefer the zone serving its --location and fall through to another bucket when that zone is down.

Two caps bound one response; a request's limit and max_bytes can only ask for less, and one record always serves:

The object cache

Cold reads (a star's fall-through reads and the Gateway's vended fetches) can route through the object cache: a per-zone pool of read-through caching proxies in front of the standard bucket, chosen per key by client-side ring hash with bounded load. It is never authoritative and may never stall a read: every cache-routed request has the ordinary direct ranged GET as its substitute, and the auditor's deletes route through it so cached bytes die with the object. The tier's routing, fallback ladder, zone scoping, and proxy instances have a page of their own.

Read plans and vending

For settled history the fleet adds nothing to the bytes' transit, so nova can hand a consumer the locations instead. ReadPlan answers a read plan for a stream range: vended entries (settled objects with the footer window and data extent to fetch) plus one open-ended residual, "from sequence X up, ask nova". It is a partition of responsibility, never a visibility claim: everything vended is settled, the residual can never hide a durable record, and the plan carries no tail. Any star serves it from one catalog read with no redirect; authorization is exactly the read action. Plans are capped at 256 entries and continue by sequence cursor.

Vending hands out those locations in one of two forms, chosen per deployment on novad:

The TTL bound is what makes a plan safe: every settled-object deletion waits out the read grace before the physical delete, so a plan cannot outlive the objects it names. Staleness is re-plan, never error: a fetch that meets a replaced revision or an expired plan re-plans from its cursor and gets the same records under new locations.

The Gateway is the plan's consumer. With --serve-plan it plans first for every read and SSE catch-up, streams vended entries up to the residual start, then opens the ordinary Read or Subscribe at exactly that sequence, for one seamless response. Its pipeline is bounded: --fetch-inflight (default 64MiB) is one byte pool over every response, and --footer-cache-bytes (default 32MiB) budgets decoded footers. Raw plans need --storage-bucket on the Gateway; without it they are proxied. See reading for the client view.

Source documents

• docs/design/024-readplan.md
• docs/design/026-object-cache.md
• docs/design/029-wake-on-ack.md
• docs/adr/0017-linearizable-reads.md
• docs/adr/0037-unsigned-range-cache-reads.md
• docs/adr/0040-subscriptions-park-on-the-leaders-wake.md