Buckets

A bucket holds streams and governs their configuration. This page covers how buckets and keys name a stream, why a stream's address and its identity are two different things, the default configuration a bucket gives every stream created in it, how buckets can create streams on first use, and how to page the bucket registry.

Every stream lives in one bucket

Every stream is created in exactly one bucket, and the bucket must already exist; nova has no implicit or reserved buckets. Tools may adopt a conventional name (nova uses default when you give none) but they create it like any other bucket.

A bucket carries three things:

its name, which is half of every stream's address;
its default stream configuration, which every new stream starts from (class, producer policy, timestamping, retention, batch, throughput);
the create flags, create_stream_on_append and create_stream_on_read, that let first use create a stream.

A bucket's name reaches into object storage too: when records settle into per-stream objects, nova writes those objects under a prefix named after the bucket. A bucket also records an opaque location, a reserved field with no routing meaning today.

Bucket naming

A stream's address is (bucket, key): the bucket names the registry entry, the key names the stream within it. The two halves follow different rules.

Bucket names

Bucket names are globally unique. The cluster keeps one flat registry of buckets, so creating a bucket claims its name until the bucket is deleted; there are no per-team or per-region bucket namespaces. That is what makes a (bucket, key) address unique.

A bucket name is 3 to 63 bytes of lowercase letters, digits, and hyphens, with no leading or trailing hyphen. Dots are not allowed. The strictness is load-bearing: bucket names embed in metadata keys and object prefixes.

Stream keys

A stream's key is its name within the bucket: 1 to 512 bytes of valid UTF-8 with no control characters. Everything else is legal and opaque; nova never parses a key.

/ is a grouping convention, not a hierarchy; S3 reads it the same way. orders, orders/, and orders/eu are three unrelated keys; creating orders/eu does not create orders. Keys are immutable; there is no rename.

Keys on the HTTP surface

A key rides the URL as one percent-encoded path segment: orders/eu is /v1/streams/{bucket}/orders%2Feu. Quote the URL so the shell does not eat the escape.

Address and ID

Every stream has an address and an ID.

The address is the pair (bucket, key), and it names at most one stream at a time: creation claims it with one atomic compare-and-set, so a race between two creates of the same address produces one stream, never two.

The ID is 16 bytes that nova assigns at creation, and no two streams ever share one, not even successive streams at the same address. Internally, everything nova stores for a stream (metadata rows, refs, settled object keys, retention markers) is keyed by the ID, not the address.

An address can be reused; an ID cannot. That is the rule to design around:

A stream recreated at the same address is a new stream, with a fresh ID and an empty history. Whatever the deleted stream left behind is keyed by its old ID, so nova reclaims it without ever confusing it with the new stream.

Every verb takes the address: the SDK speaks client.Address{Bucket, Key}, the HTTP surface addresses streams in the URL path, and the nova CLI writes an address as nova://bucket/key. The ID never goes in — it comes back in outputs (create and get responses, log lines, settled-object keys) as the handle that correlates a stream with its internal artifacts.

The default stream configuration

A bucket carries a default stream configuration: a complete set of stream options that every stream created in the bucket starts from, explicit creates and autocreates alike. A create request names the fields it wants to set itself; every field it leaves out is copied from the bucket's defaults:

Field	Wire name on the bucket	Values
Class	`default_class`	`standard`, `express`
Producer policy	`default_producer_policy`	`any`, `fenced`
Timestamping	`default_timestamping_mode`, `default_timestamping_uncapped`	`client-prefer`, `client-require`, `arrival`; uncapped true/false
Retention	`default_retention_max_age_ms`, `default_retention_max_bytes`	zero = unbounded
Batch	`default_batch_max_bytes`, `default_batch_max_records`, `default_batch_max_delay_ms`	flush triggers
Throughput	`default_throughput_max_bytes_per_second`	zero = unlimited

The defaults are always concrete: a field the bucket itself leaves unset takes nova's domain default when the bucket is created (Standard, any, client-prefer, unbounded retention, an 8 MiB / 250 ms batch, unlimited throughput). Domain defaults only ever fill the bucket's configuration; they never reach a stream directly.

The copy happens once, at creation (copy-at-create). Changing the defaults with UpdateBucket affects only streams created afterwards; existing streams keep the configuration they were created with. Two streams created in one bucket by different doors with no overrides are indistinguishable.

Zero means unset on the wire

The scalar defaults (retention age and bytes, throughput, timestamping_uncapped) arrive as zero whether omitted or set, so a create can override a bucket's retention to another bound but not back to unbounded, nor turn uncapped off. Put the permissive value in the bucket and the strict one in the stream, not the other way round — or create first and then reconfigure, whose fields carry presence and can set a bound to zero.

Create on first use

Set create_stream_on_append and the first append to a missing key creates the stream from the bucket's defaults, exactly once, under the address CAS; a lost race reads the winner and joins. create_stream_on_read does the same for reads and subscriptions (the stream is created empty), so a subscriber may arrive before its producer.

A bucket that creates streams on first append

nova bucket create apps --create-on-append

The same over HTTP, then an append that creates its stream

curl -s -X POST localhost:8080/v1/buckets/apps -d '{"create_stream_on_append":true}'
curl -s -X POST localhost:8080/v1/streams/apps/orders%2Feu/records \
  -d '{"records":[{"body":"hi"}]}'

Autocreate creates streams, never buckets. An explicit CreateStream on a taken address fails with AlreadyExists rather than returning the existing stream, since returning it would misrepresent its options. Creates carry an optional creation_token; a retry with the same token converges on the existing record as success.

Observing verbs never create: GetStream, CheckTail, Fence, Trim, and ReadPlan resolve the address and report not-found if it is missing.

Listing buckets

ListBuckets pages the registry in name order, up to 1000 buckets per page. The cursor is a bucket name: start_after is the last name of the prior page, and the response's next is empty on the last page. There is no name-prefix filter. Because the page spans the whole registry, the caller needs whole-fleet list authority (the *:* grant); a credential scoped to one bucket cannot list buckets at all. See authentication.

Browse with nova

nova ls                          # buckets, one name per line
nova bucket list --limit 50      # the same, with paging flags
nova bucket list --cursor apps   # the page after "apps"

The same over HTTP

curl -s 'localhost:8080/v1/buckets?limit=50'

Listing the streams inside a bucket is a different verb with a different scope; see listing streams.

Bucket lifecycle

Verb	What it does
`CreateBucket`	Registers the bucket with its flags and default stream configuration. Idempotent with a `creation_token`.
`GetBucket`	Returns the record as stored, defaults included.
`UpdateBucket`	Replaces the caller-writable config: flags act immediately, the defaults govern future streams.
`DeleteBucket`	Removes the bucket, but only when no streams live in it; otherwise `FailedPrecondition`.
`ListBuckets`	Pages the registry by name.

nova bucket verbs

nova bucket create <name> [--create-on-append] [--create-on-read] [--default-class standard|express] [--location ...]
nova bucket get <name>
nova bucket list [--limit N] [--cursor <name>]
nova bucket delete <name>

In Go, client.Bucket carries Name, Location, CreateOnAppend, CreateOnRead, and Defaults (a client.StreamOptions), and Client.EnsureBucket is create-or-join for tools that want a conventional bucket to exist.

Source documents

docs/design/020-stream-addressing.md
proto/nova/v1/dataplane.proto, proto/nova/v1/metadata.proto
client/buckets.go, client/address.go

On this page