novadocs
Operations

Operations

The admin dashboard

A read-only view of the whole cluster from any star — stars, streams, buckets, credentials, audit queues, and a record reader

Every star can serve a browser dashboard of the whole cluster: which streams exist and who leads them, how loaded each star is, whether reclamation is keeping up, and, behind its own flag, the records themselves. It is read-only by construction and rides the ops listener with the trust stance of /metrics.

Enabling it

--admin-ui (NOVAD_ADMIN_UI, default on) serves the dashboard at /admin/ on the metrics listener (--metrics-listen). No metrics listener, no dashboard. In the compose stack it is http://localhost:9191/admin/; any star shows the same cluster.

FlagDefaultEffect
--admin-uionserve the dashboard and its JSON bridge at /admin/
--admin-ui-readeronmirror the admin reader's record read on the bridge, the one endpoint that returns record bodies; off keeps the metadata dashboard while only the authenticated gRPC plane serves records
--metrics-listenemptythe ops listener; empty removes the whole surface

The ops listener carries no authentication of its own: it is trusted-network tooling, never internet-exposed. Mutations go through nova over the authenticated gRPC admin plane; with --auth basic, --admin-username and NOVAD_ADMIN_PASSWORD set that plane's credential, and nova --admin-user/--admin-password present it. See authentication.

The admin bridge

Every datum the dashboard shows is first a proto-typed query RPC on the admin plane. The ops mux carries a thin JSON bridge (/admin/api/{Method}) that decodes the request into the same proto message, dispatches through the gRPC service descriptor into the same service implementation, and encodes the answer with protojson. One handler, two wires: nova and any tooling get every endpoint for free, and the browser's types are generated from the same proto.

The bridge mirrors exactly these RPCs: Health, ListStars, GetClusterOverview, ListStreamSummaries, ListBucketSummaries, GetStreamDetail, GetStreamLiveStats, ListAuditQueues, PeekAuditQueue, ReadStreamRecords, and ListCredentials. Drain, Decommission, the credential mutations, and everything else on the admin plane are absent from the list, so the bridge is structurally incapable of mutation: there is no code path from a browser to a write.

The bridge from the shell
curl -s localhost:9191/admin/api/GetClusterOverview -d '{}'
curl -s localhost:9191/admin/api/ListAuditQueues -d '{}'

Where the data comes from

  • Global metadata (descriptors, refs, trim watermarks, due-queue entries) is read from the shared metastore, so any star can serve it.
  • Node aggregates come from each star's node stats record: a small, fixed-size, session-ephemeral record every star refreshes on its stats tick (leaders, append and read bytes/sec, saturated, stranded, draining, drained, utilization and its axis, metastore p99, version). The cluster overview is one bounded prefix scan over the star count; a star whose record is absent or stale is shown as such: partial data is marked, never hidden.
  • Per-stream live stats (bytes, hot-read bytes, the tail) are never published to the metastore, because those maps would grow with led-stream count. A stream's detail page fetches them on demand with one RPC routed to the leader; a non-leader proxies one hop. Leader unreachable → the durable metadata renders and the live section is marked unavailable.

Every list is paginated and every scan bounded; a count that stops at a cap says so (N or cap+). The browser polls unary RPCs (about 2s on the overview, 5s on detail pages); the server holds no dashboard state.

Pages

PageShows
Overviewthe star table (liveness plus stats records), cluster totals, and an auditor health strip
Streamspaginated descriptors named by address (nova://bucket/key): id, class, producer policy, leader; a by-bucket filter; a live/idle filter over the page that came back
Stream detailthe descriptor, refs and settled catalog, trim watermark, pending audit jobs for this stream (via the by-stream index), live leader stats, and the admin reader
Bucketsthe registry rows as scopes (nova://bucket/): create-on-append flags, the default stream configuration every new stream starts from (class, producer policy, retention, throughput, batch, timestamping), tombstones in flight; each row opens its defaults and links to the bucket's streams
Auditsper-queue summaries (oldest-ready age and a bounded depth) and a bounded peek into each queue
Credentialsthe api-key registry: key id, active or revoked, the scope rendered grant by grant, created, never a secret or a hash; a star without a registry says so

What "idle" means here

The streams page labels a stream Idle when no writer lease is held: nothing serves it right now. In the glossary that state is vacant; idle names the trigger that leads to it (an Any-policy leadership whose last producer closed and whose idle timeout is running). The console uses the plainer word for operators; the distinction matters only when you read the design docs. The leader column is a per-row lease lookup, so vacancy is never a server-side scan, and the footer discloses that the live/idle count is page-scoped.

The admin reader

The stream detail page carries a record-consuming panel, collapsed to a Start button until used. Unlike every other section it is a data-plane consumer riding admin transport, not a metadata observer.

  • Start anywhere. From a sequence number, a timestamp, or the tail (the default: the last ~50 records render, then it follows). Cursor and timestamp starts catch up at full page speed, then follow. The start spec mirrors into the URL, so a refresh or a shared link reopens the same view.
  • Vacant streams stay asleep. Read routing is the core's own: a vacant stream serves committed state locally with no lease acquired and no wake; a wholly-cold page serves on any star; only a read touching a foreign-leased hot window proxies one hop to the holder, never chaining. A freshly-vacated stream's unconverged residue bounces the read until settlement converges; the reader retries and says so.
  • A cursor loop, not a stream. The browser polls the unary ReadStreamRecords about once a second while following, back-to-back while catch-up pages return full, and advances the cursor itself. Stop is a frozen cursor; scrolling off the bottom pauses consumption; leaving the page ends the loop. The server holds no reader state.
  • Bounded end to end. One page is at most 500 records or 1 MiB; the browser's display ring keeps at most 1,000 records or 16 MiB per tab, dropping the oldest with a disclosed count; bodies collapse past 4 KiB and render as text, pretty-printed JSON, or hex by inspection: always switchable, always opaque bytes on the wire.

Because this endpoint returns user data rather than metadata, it has its own switch: --admin-ui-reader=false removes it from the bridge while the authenticated gRPC admin plane keeps serving the RPC.

Bounds

AccumulationBound
node stats recordfixed-size aggregates only, never per-stream maps
stats publishone small write per star per tick
cluster overviewone prefix scan over the star count
stream list, queue peek, pending-jobs listexplicit limit plus cursor
queue depthcounted to a cap, rendered cap+
live-stats fan-outone RPC to one leader, on demand
reader page / display ring500 records or 1 MiB; 1,000 records or 16 MiB per tab

Source documents

  • docs/design/008-admin-ui.md
  • proto/nova/v1/adminplane.proto
  • internal/adminui/adminui.go
  • web/admin/src/routes/

On this page