HTTP API

The HTTP surface is the data plane re-encoded as JSON under /v1/…, with subscriptions as server-sent events. You reach it through the Gateway on the internet or through the data bridge inside a trusted network; the routes and envelopes are identical on both.

Where it is served

• The Gateway (novagateway, listening on :8080 by default) is the internet-facing tier: TLS termination, edge token verification, per-bucket hosts, CORS, and limits. It dispatches into the fleet and absorbs leader redirects, so one hostname is the whole topology. In the compose stack it is localhost:8080. How the tier is built (the trust boundary, the middleware stack, plan-driven serving) is its architecture page.
• The data bridge is a listener (--http-listen, off by default) on every star, every running novad, for callers inside the trust domain that cannot speak gRPC. It serves the same routes under the star's own auth mode and TLS identity; a stream led elsewhere is proxied one hop to its leader. In the compose stack the bridges are localhost:7791 through 7793.

An HTTP caller never sees a redirect on either host. Producer sessions have no HTTP form: appends over HTTP are one-shot.

URL grammar

Streams live at /v1/streams/{bucket}/{key} and buckets at /v1/buckets/{bucket}. The key is one percent-encoded path segment: keys may contain /, so orders/eu travels as orders%2Feu. Quote the URL so the shell does not eat the escape, and make sure any proxy in front passes escaped slashes through untouched.

On a Gateway configured with a DNS zone, two host styles serve the same grammar:

A bucket host serves only its streams; bucket administration rides the api hostname.

Authentication

Send the token as a bearer: Authorization: Bearer <jwt>. Tokens travel in that header only, never in query parameters. The one anonymous route is the exchange, which authenticates with the api-key pair in its body:

curl -s localhost:8080/v1/token/exchange \
  -d '{"key_id":"ck_app","secret":"...","bucket":"apps","ttl_seconds":900}'

The answer is {"token":"…","expires_at_unix_ms":…}. The dev compose stack runs with auth off, so the examples below carry no header; see authentication for the token grammar.

Buckets and streams

curl -s -X POST localhost:8080/v1/buckets/apps \
  -d '{"create_stream_on_append":true,"default_class":"express"}'

curl -s -X POST localhost:8080/v1/streams/apps/orders%2Feu \
  -d '{"class":"standard","producer_policy":"any","retention_max_age_ms":604800000}'

Both answer 201 with the record as stored (a retry carrying the same creation_token answers 201 again). An empty body means "every default". The stream config fields are class (standard | express), producer_policy (any | fenced), retention_max_age_ms, retention_max_bytes, batch_max_bytes, batch_max_records, batch_max_delay_ms, throughput_max_bytes_per_second, timestamping_mode (client-prefer | client-require | arrival), timestamping_uncapped, and creation_token. A bucket body carries location, create_stream_on_append, create_stream_on_read, the same fields prefixed default_, and creation_token.

GET /v1/streams/apps/orders%2Feu returns the record plus fencing_token (empty when unfenced). PATCH with a body naming only the fields to change reconfigures the stream and answers 200 with the record as it now stands: a field absent from the body is untouched, a present one is set ({"retention_max_age_ms":0} returns retention to unbounded); a change to class, producer policy, timestamping, throughput, or batch hands a live stream off, so its producers reconnect once. DELETE answers 204. GET /v1/streams/{bucket} pages a bucket's streams with ?start_after=<cursor>&limit=N, answering {"streams":[…],"next":"…"} (there is no listing across buckets); buckets page the same way at GET /v1/buckets. PUT /v1/buckets/{bucket} replaces the caller-writable config whole; DELETE succeeds only on an empty bucket.

Appending

POST …/records is a one-shot append: one request, one durable batch.

curl -s localhost:8080/v1/streams/apps/orders%2Feu/records \
  -d '{"records":[
        {"body":"hello","headers":[{"name":"content-type","value":"text/plain"}]},
        {"body_b64":"AAEC","timestamp_ms":1724500000000}
      ]}'

Each posted record names its own encoding: body is a UTF-8 string, body_b64 is base64 bytes; give exactly one of the two, or the request is 400. Headers are objects with exactly one of name / name_b64 and one of value / value_b64 (an omitted value is the empty value). timestamp_ms is optional and honored per the stream's timestamping mode. A batch is at most 65,536 records, each at most 1 MiB of content, 100 MiB in total.

The answer mirrors the wire: {"start":{"seq_num":0,"timestamp_ms":…},"end":{"seq_num":2,…},"tail":{"seq_num":2,…}}; end and tail are exclusive.

Two optional envelope fields: match_seq_num makes the append conditional on the batch's first record receiving exactly that sequence number (a miss answers 412 with the committed tail in the body and appends nothing); fencing_token presents a token on a fenced stream (a mismatch answers 412 with the installed fencing_token). The bridge holds no idempotency state, so a plain append retried after an ambiguous outcome may duplicate; use match_seq_num, or read the tail and decide.

curl -s localhost:8080/v1/streams/apps/orders%2Feu/records \
  -d '{"records":[{"body":"third"}],"match_seq_num":2}'

Reading

GET …/records answers one page immediately, possibly empty; it never long-polls.

curl -s 'localhost:8080/v1/streams/apps/orders%2Feu/records?seq_num=0&limit=100'
curl -s 'localhost:8080/v1/streams/apps/orders%2Feu/records?timestamp_ms=1724500000000'
curl -s 'localhost:8080/v1/streams/apps/orders%2Feu/records?tail_offset=10'
curl -s localhost:8080/v1/streams/apps/orders%2Feu/records/tail

Exactly one start is required: seq_num, timestamp_ms (the first record stamped at or after it), or tail_offset (that many records before the tail). Knobs: limit (clamped to 8192), max_bytes (clamped to 8 MiB; at least one record always returns), until_ms (exclusive upper time bound), clamp=true (pull a start below the trim point or past the tail into range instead of 410/empty), and format.

The response is {"records":[{"seq_num":0,"timestamp_ms":…,"body":"hello","headers":[…]}],"tail":{…}}. By default (format=raw) a body that is valid UTF-8 rides body; one that is not arrives as body_b64, per record, per header field, never lossy. format=base64 carries every body and header field base64-encoded in body, name, and value. …/records/tail answers the committed tail as a position: {"seq_num":N,"timestamp_ms":…}, N being the next sequence number.

Subscribing with SSE

GET …/records/subscribe streams batches as server-sent events and pends at the tail.

curl -N 'localhost:8080/v1/streams/apps/orders%2Feu/records/subscribe?seq_num=0'

id: 3
data: {"records":[{"seq_num":0,…},{"seq_num":1,…},{"seq_num":2,…}],"tail":{"seq_num":3,…}}

: hb

event: auth-expired
data: {}

Each event's id is the next sequence number, so a reconnect sending Last-Event-ID: 3 resumes exactly where it left off: star drains and Gateway restarts are invisible. Last-Event-ID wins over the query's seq_num / timestamp_ms; without either, the request is 400. Comment lines (: hb, every 15s) are heartbeats that defeat load-balancer idle timeouts. Novad hangs up on a consumer that stops reading for 60s. When the token backing the subscription expires, the stream ends with an auth-expired event: refresh, reconnect, resume with Last-Event-ID. The stream is pull-paced (a slow consumer slows novad, nothing queues), and each host caps concurrent SSE connections (4096 by default), shedding with 503.

In browsers, use a streaming fetch rather than EventSource, which cannot send the Authorization header.

Trim and fence

curl -s -X POST localhost:8080/v1/streams/apps/orders%2Feu/trim -d '{"seq_num":1000}'
curl -s -X POST localhost:8080/v1/streams/apps/orders%2Feu/fence -d '{"fencing_token":"writer-2"}'

Trim takes exactly one of seq_num or timestamp_ms and answers 204; the watermark only advances. Fence installs, rotates, or, with an empty or absent token, clears the stream's fencing token, superseding every live session the new token no longer admits, and answers {"tail":{…}}, the admission boundary.

Errors

Every error is {"error":"<reason>"} under a status from a closed set; the two 412s add a machine field.

Gateway-only behavior

• GET /healthz answers 200 outside the whole middleware stack: the load balancer's target check.
• CORS is default-deny with an explicit origin allowlist; a preflight from an allowed origin gets GET, POST, DELETE, OPTIONS and the headers Authorization, Content-Type, Last-Event-ID; a foreign origin's preflight is 403.
• Limits ride the token as claims (nova_aux.limits: rps, connections, read_Bps, write_Bps) and are enforced per Gateway instance; an over-budget request is 429, an SSE stream at an exhausted byte budget pauses rather than dropping.
• Plan-driven serving (--serve-plan, on by default) lets the Gateway fetch settled history straight from object storage instead of proxying it through the fleet; --serve-plan=false proxies everything.

The novagateway binary today runs verify-off, path-style on any host, without CORS or limits (the dev posture); its flags cover listening, TLS termination, the fleet addresses, storage identity for raw vending, the object cache, and the fetch and footer budgets. Edge verification, bucket hosts, CORS, and limits are Gateway middleware that a deployment arms in its wiring.

Source documents

• docs/design/021-http-surface.md
• docs/design/022-gateway.md
• internal/transport/http/dataplane/
• internal/gateway/
• deploy/docker/README.md