novadocs
Reference

Reference

novad reference

Every novad flag, its NOVAD_* environment variable and default, grouped by concern, plus the YAML config file

novad is the serving process (a running one is a star) and, unless told otherwise, also the background auditor. Every setting is a flag with a NOVAD_* environment variable and a YAML key of the same name; this page lists them all, grouped by concern, with defaults copied from novad --help. A default of 0 or 0s means "use the built-in value" named in the meaning column.

novad [global options]

Resolution order for every setting: a command-line flag beats its environment variable, which beats the config file, which beats the built-in default. Secret material (S3 secret key, HMAC secrets, passwords, peer tokens, the root secret) should come from the environment or the config file, never from argv, where it is visible in process listings; two of them (--admin-password and --peer-service-token) print no help text for exactly that reason.

Config file and dev mode

FlagEnvDefaultMeaning
--configNOVAD_CONFIG—YAML config file; flags and NOVAD_* env override it
--devNOVAD_DEVfalseSingle-process playground: in-memory backends, data vanishes on exit
--version, -v——Print the version

Outside --dev, the process refuses to start without --oxia, --oxia-naming, --s3-bucket, and an identity: --node-id, or --dir to read or mint one.

Serving and limits

The client-plane caps. A request asking for more than a cap is clamped to it; an append exceeding one is refused.

FlagEnvDefaultMeaning
--max-read-bytesNOVAD_MAX_READ_BYTES0Bytes cap on one read response's payload; a request's max_bytes clamps to it; zero uses 8 MiB
--max-read-recordsNOVAD_MAX_READ_RECORDS0Record cap on one read response; a request's limit clamps to it; zero uses 8192
--max-record-bytesNOVAD_MAX_RECORD_BYTES0Bytes cap on one appended record; zero uses 1 MiB
--max-batch-bytesNOVAD_MAX_BATCH_BYTES0Bytes cap on one append batch's payload total; zero uses 100 MiB
--max-batch-recordsNOVAD_MAX_BATCH_RECORDS0Record cap on one append batch; zero uses 65536
--max-producer-sessionsNOVAD_MAX_PRODUCER_SESSIONS0Cap on concurrent producer sessions per star; opens past it shed with ResourceExhausted; zero uses 4096
--grpc-stream-window-kibNOVAD_GRPC_STREAM_WINDOW_KIB0Pin the server's HTTP/2 flow-control windows: KiB per stream (64 minimum, 65536 maximum), 4× per connection. Inbound bytes buffered ahead of the handlers are bounded per stream, so sessions × window is the transport's memory plane; zero pins the 64 KiB protocol floor
--net-capacityNOVAD_NET_CAPACITY0Sustained network capacity in bytes/sec, stamped by the operator: the network saturation axis's denominator; zero leaves the axis off
--memory-ceilingNOVAD_MEMORY_CEILING0Retained-memory budget in bytes, stamped by the operator from the pod limit with margin: the memory saturation axis's denominator; zero leaves the axis off

Identity and placement

FlagEnvDefaultMeaning
--dirNOVAD_DIR—Directory holding the star's identity file; optional when --node-id pins it
--node-idNOVAD_NODE_ID—Pin the star's StarID (UUID); default: read or mint <dir>/node-id
--locationNOVAD_LOCATION—Where this star runs (an availability zone in cloud terms): the placement spread label, and the express zone its reads prefer
--advertiseNOVAD_ADVERTISE—Address announced to the fleet; default: the listen address (hostname when unspecified). Clients and the Gateway dial redirect targets verbatim, so it must resolve from their network position

Listeners and TLS

One gRPC listener serves every plane (data, admin, and peer traffic) under one auth mode and one TLS identity.

FlagEnvDefaultMeaning
--listenNOVAD_LISTEN:7780gRPC listen address for all services (the advertised address)
--http-listenNOVAD_HTTP_LISTEN—HTTP address for the data bridge; empty disables it
--metrics-listenNOVAD_METRICS_LISTEN—HTTP address to serve Prometheus /metrics (also /healthz, /readyz, and the admin UI); empty disables metrics
--admin-uiNOVAD_ADMIN_UItrueServe the admin dashboard and its read-only JSON bridge at /admin/ on the metrics listener; the whole surface vanishes without --metrics-listen
--admin-ui-readerNOVAD_ADMIN_UI_READERtrueMirror the admin reader's record read on the bridge, the dashboard's one endpoint that returns record bodies; off keeps the metadata dashboard while the authenticated gRPC plane alone serves records
--tls-certNOVAD_TLS_CERT—TLS serving certificate; empty serves plaintext
--tls-keyNOVAD_TLS_KEY—TLS private key; required with --tls-cert
--tls-caNOVAD_TLS_CA—CA bundle for verifying peers on outbound dials; empty uses the system roots
--tls-client-caNOVAD_TLS_CLIENT_CA—CA bundle for verifying client certificates; required by auth mode mtls

Auth

One mechanism applies to every caller, clients and peers alike. Modes jwt and basic carry credentials on the wire, so they require TLS or an explicit --insecure-auth.

FlagEnvDefaultMeaning
--authNOVAD_AUTHoffAuth for every caller: off, mtls, jwt, or basic
--insecure-authNOVAD_INSECURE_AUTHfalseExplicitly accept credentials over plaintext (TLS terminated elsewhere)
--jwt-hmac-secret-fileNOVAD_JWT_HMAC_SECRET_FILE—Verify tokens against this shared secret (HS256 only); one jwt source exactly
--jwt-hmac-secretNOVAD_JWT_HMAC_SECRET—The shared secret itself, inline, the diskless variant; secret material, prefer the env
--jwt-hmac-keysNOVAD_JWT_HMAC_KEYS—Named key set, id:secret comma-separated: the first signs, all verify, minted tokens stamp their kid; rotation is prepend-and-roll; secret material, prefer the env
--jwt-jwks-urlNOVAD_JWT_JWKS_URL—Verify tokens against this JWKS endpoint (asymmetric algorithms only)
--jwt-oidc-issuerNOVAD_JWT_OIDC_ISSUER—Discover the JWKS from this OIDC issuer (asymmetric algorithms only)
--jwt-audienceNOVAD_JWT_AUDIENCE—Expected aud claim; empty skips the audience check
--admin-usernameNOVAD_ADMIN_USERNAME—Basic-auth username; the password comes from NOVAD_ADMIN_PASSWORD or the config file
--admin-passwordNOVAD_ADMIN_PASSWORD—Basic-auth password; env or config file only
--peer-service-token-fileNOVAD_PEER_SERVICE_TOKEN_FILE—File holding the token this star presents on peer dials (jwt with a JWKS/OIDC source only)
--peer-service-tokenNOVAD_PEER_SERVICE_TOKEN—The token itself; env or config file only
--root-credential-fileNOVAD_ROOT_CREDENTIAL_FILE—nova-generated seed converged into the credential registry at boot
--root-credentialNOVAD_ROOT_CREDENTIAL—The seed's JSON passed inline (holds only the hash), the diskless variant
--root-key-idNOVAD_ROOT_KEY_ID—The pair form of the root seed: the key id; comes with NOVAD_ROOT_SECRET, the plainest bootstrap for compose runs
--root-secretNOVAD_ROOT_SECRET—The pair's secret, hashed at boot; secret material, prefer the env

Validation at startup:

  • mtls needs --tls-client-ca and the server pair.
  • jwt needs exactly one source among --jwt-hmac-secret-file, NOVAD_JWT_HMAC_SECRET, NOVAD_JWT_HMAC_KEYS, --jwt-jwks-url, --jwt-oidc-issuer. With a JWKS or OIDC source the star cannot self-mint its peer credential, so --peer-service-token-file (or NOVAD_PEER_SERVICE_TOKEN) is required.
  • basic needs --admin-username and NOVAD_ADMIN_PASSWORD.
  • The root credential is one seed: set at most one of --root-credential-file, --root-credential, or the NOVAD_ROOT_KEY_ID/NOVAD_ROOT_SECRET pair (which come together).

How the modes fit clients and the Gateway is the authentication guide's subject.

Metastore (oxia)

FlagEnvDefaultMeaning
--oxiaNOVAD_OXIA—oxia service address (required)
--oxia-namingNOVAD_OXIA_NAMING—NATURAL-sorted oxia service address for the naming keyspace (required)
--oxia-session-timeoutNOVAD_OXIA_SESSION_TIMEOUT0soxia session timeout, the fleet's failure-detection floor: a dead star's leases and liveness survive this long, and failover waits it out; zero uses the client default (15s); keep it above worst-case process stalls
--oxia-write-windowNOVAD_OXIA_WRITE_WINDOW0Write batches in flight per oxia shard, pipelined, ordered even across failovers; zero uses the client default (4), one disables pipelining
--oxia-max-batch-kibNOVAD_OXIA_MAX_BATCH_KIB0Byte cap on one oxia write batch in KiB; with one batch in flight per shard this bounds per-star row throughput; zero uses the client default (128)
--oxia-records-namespaceNOVAD_OXIA_RECORDS_NAMESPACE—Namespace for durable records; default: the client default
--oxia-work-namespaceNOVAD_OXIA_WORK_NAMESPACE—Namespace for work queues
--oxia-ephemeral-namespaceNOVAD_OXIA_EPHEMERAL_NAMESPACE—Namespace for ephemerals (liveness, leases)

Object storage and the Standard write path

FlagEnvDefaultMeaning
--s3-bucketNOVAD_S3_BUCKET—Object storage bucket (required)
--s3-endpointNOVAD_S3_ENDPOINT—Custom S3 endpoint (e.g. MinIO); empty uses AWS S3
--s3-regionNOVAD_S3_REGION—S3 region
--s3-access-keyNOVAD_S3_ACCESS_KEY—S3 access key; leave both keys empty to use the SDK default chain (env, shared config, IAM role/IRSA)
--s3-secret-keyNOVAD_S3_SECRET_KEY—S3 secret key (prefer the env or the config file over argv)
--s3-path-styleNOVAD_S3_PATH_STYLEfalseUse path-style addressing (MinIO and most non-AWS endpoints)
--s3-attempt-timeoutNOVAD_S3_ATTEMPT_TIMEOUT0sCap on one S3 request attempt (e.g. 2s): a straggling attempt is cut at the deadline and transparently retried, sequential hedging for the PUT tail; size it well above the largest routine PUT's p50; zero leaves attempts unbounded
--s3-bundle-size-mibNOVAD_S3_BUNDLE_SIZE_MIB0Pooled payload MiB at or above which a Standard bundle flushes immediately; zero uses 8; bigger bundles amortize PUT cost, smaller ones cut time-to-durable
--s3-bundle-batch-intervalNOVAD_S3_BUNDLE_BATCH_INTERVAL0sDeadline no pooled Standard byte waits past before its bundle flushes regardless of size; zero uses 200ms, the ack-latency bound at low traffic
--s3-parallel-uploadsNOVAD_S3_PARALLEL_UPLOADS0Pipelined Standard slot PUTs per star; acknowledgements stay ordered regardless; zero uses 128
--s3-memory-buffer-mibNOVAD_S3_MEMORY_BUFFER_MIB0MiB of appends pooled but not yet flushed on the Standard write path before producers block; zero uses 256, the admission buffer that absorbs bursts between flushes
--s3-chain-backlog-flightsNOVAD_S3_CHAIN_BACKLOG_FLIGHTS0Flights the Standard chain may hold un-absorbed (the backlog budget); appends block at the bound until the absorb's checkpoint catches up. Bounds leader tail memory, takeover settlement time, and chain storage together; zero uses 1024

Express

A non-empty bucket set enables the Express class on this star. Zone labels match the stars' --location spelling; when a label is an AZ id (use1-az4) it must match the id embedded in its directory bucket's name, and directory buckets refuse path-style addressing.

FlagEnvDefaultMeaning
--s3-express-bucketsNOVAD_S3_EXPRESS_BUCKETS—Express bucket set as comma-separated zone=bucket pairs (one per AZ); non-empty enables Express, whose reads prefer the zone serving this star's --location
--s3-express-ack-quorumNOVAD_S3_EXPRESS_ACK_QUORUM0Express bucket copies that must confirm before an append acks; zero uses 2
--s3-express-publish-deadlineNOVAD_S3_EXPRESS_PUBLISH_DEADLINE0sDeadline for an express flight to confirm its copy quorum; past it appends fail fast and producers retry; zero uses 2s
--s3-express-parallel-uploadsNOVAD_S3_EXPRESS_PARALLEL_UPLOADS0Concurrent express bundle PUTs (at most 256); each fans to every bucket in the set; zero uses 16
--s3-express-memory-buffer-mibNOVAD_S3_EXPRESS_MEMORY_BUFFER_MIB0MiB of bundle bytes buffered for express fanout before admission blocks (at most 16384); zero uses 32

Absorb and the tail window

FlagEnvDefaultMeaning
--absorb-fanoutNOVAD_ABSORB_FANOUT0Concurrent per-stream metadata commits the absorb fans across a batch (refs, covering segments, liveness); the absorb's drain rate at high stream density scales with it, bounded so one star's bursts cannot monopolize the shared metastore; zero uses 4096
--absorb-pass-intervalNOVAD_ABSORB_PASS_INTERVAL0sPacing floor on the absorb pass cadence (e.g. 5s): released flights pool behind it and one paced pass commits the window, amortizing each stream's metastore ops to one per interval, at the cost of that much absorb lag (non-leader read visibility, failover settlement). A publish stalling on the backlog budget breaks the floor. Zero absorbs on every release
--tail-window-cache-mibNOVAD_TAIL_WINDOW_CACHE_MIB0MiB of acked flight payloads the leader retains in memory (the tail window cache), serving hot reads without object storage until the absorb's checkpoint retires them; one budget shared across every class; zero uses 128

Auditor and reclamation

The auditor runs embedded in every star by default. --audit-only runs the same image as a reclamation pod with no serving plane and no star registration; --no-audit turns it off on a serving star. Setting both is a configuration error. Sizing guidance is in auditor tuning.

FlagEnvDefaultMeaning
--no-auditNOVAD_NO_AUDITfalseDisable the background auditor on this star
--audit-onlyNOVAD_AUDIT_ONLYfalseRun only the background auditor: no engine, no serving plane, no star registration
--auditor-s3-budgetNOVAD_AUDITOR_S3_BUDGET0Background object-store requests per second the auditor may issue; the settle rewrite moves every transit byte the write path commits, so size this to sustain ingest or the rewrite backlog ages; zero uses 50
--auditor-rewrite-mibNOVAD_AUDITOR_REWRITE_MIB0MiB per second the auditor's rewrites may move through this star: bytes assembled by ranged GETs and re-PUT (server-side copies are free of it); size it with the S3 budget to sustain ingest; zero uses 64
--auditor-workersNOVAD_AUDITOR_WORKERS0Concurrent background reclamation jobs (rewrites, teardown walks, GC): the throughput dial for metastore-latency-bound work; zero uses 2
--auditor-markers-per-eraNOVAD_AUDITOR_MARKERS_PER_ERA0Ripe batch markers one dispatched settle job gathers as an era: a contiguous run of one chain's slots whose bundles each download once for every stream they carry; zero or one settles one marker per job
--auditor-member-parallelNOVAD_AUDITOR_MEMBER_PARALLEL0Concurrent member settles inside one batch walk; a chain's era runs as one job, so serial member latency would cap its settle rate below the chain's production rate; zero uses 8
--auditor-fetch-cache-dirNOVAD_AUDITOR_FETCH_CACHE_DIR—Scratch directory enabling the settle's whole-object fetch cache: each transit bundle downloads once and every consumer reads the file, replacing per-extent ranged GETs. Startup wipes the directory's contents: point it at dedicated scratch (a Kubernetes emptyDir), never at data; empty keeps the per-range strategy
--auditor-fetch-cache-mibNOVAD_AUDITOR_FETCH_CACHE_MIB0MiB of scratch the fetch cache may keep resident; a read the budget cannot admit falls back to a ranged GET; zero uses 40960
--auditor-fetch-parallelNOVAD_AUDITOR_FETCH_PARALLEL0Concurrent background downloads warming the fetch cache; zero uses 64
--auditor-orphan-graceNOVAD_AUDITOR_ORPHAN_GRACE0sAge before an unresolved write intent is judged, and the bound the teardown's final walk waits out; zero uses 1h; keep it above the slowest legitimate commit
--auditor-read-graceNOVAD_AUDITOR_READ_GRACE0sReader-safety window between an object's last tombstone and its physical delete; zero uses 10m; keep it above twice any reader's cached-footer lifetime

Object cache

A non-empty instance list routes the serving lane's cold reads and the audit lane's deletes through the shared object cache; every failure falls back to the direct client. A contradictory cache config refuses to boot rather than silently serving zero hits.

FlagEnvDefaultMeaning
--object-cache-instancesNOVAD_OBJECT_CACHE_INSTANCES—Comma-separated object-cache instance base URLs (http://host:port; cached traffic is plain HTTP); empty keeps all requests direct
--object-cache-load-factorNOVAD_OBJECT_CACHE_LOAD_FACTOR1.25Bounded-load ceiling over the mean in-flight claims per instance; a hot key spills its overflow to the next instance at the ceiling; must exceed 1
--object-cache-connect-budgetNOVAD_OBJECT_CACHE_CONNECT_BUDGET1sDial budget for one cache attempt; a breach charges the instance and the request goes direct
--object-cache-first-byte-budgetNOVAD_OBJECT_CACHE_FIRST_BYTE_BUDGET5sRequest-to-first-byte budget for one cache attempt, sized above direct-S3 p99 so S3's own slowness is not misread as a sick instance; a breach charges the instance and the request goes direct

Read plans

FlagEnvDefaultMeaning
--readplan-vendingNOVAD_READPLAN_VENDING—How read plans vend settled-object locations: presigned (short-lived URLs; needs the store's presign capability; the default when empty) or raw (bucket + key for trust-domain consumers fetching under their own storage identity)
--readplan-ttlNOVAD_READPLAN_TTL0sRead-plan validity: presigned URL expiry, and the advisory fetch bound for raw consumers; zero uses 5m; must be at most half the auditor read grace

Settle timing

The two timers that bound how long a committed byte stays in transit form before it settles into a per-stream object: the linger after a stream crosses the size trigger, and the per-class age cap.

FlagEnvDefaultMeaning
--denorm-output-cap-mibNOVAD_DENORM_OUTPUT_CAP_MIB0MiB cap on one settle output object; an over-cap transit window splits at row boundaries; zero uses 256
--denorm-lingerNOVAD_DENORM_LINGER0sDelay from a Standard stream crossing the size trigger to its settle round; it batches crossings so outputs grow past the trigger; zero uses 5m
--denorm-age-capNOVAD_DENORM_AGE_CAP0sHard bound on a Standard-class byte's transit residency before its batch settles; zero uses 24h
--express-denorm-lingerNOVAD_EXPRESS_DENORM_LINGER0sThe size-trigger linger for Express streams; zero uses 5m
--express-denorm-age-capNOVAD_EXPRESS_DENORM_AGE_CAP0sHard bound on an Express-class byte's transit residency; express storage bills for the transit window, so keep this short; zero uses 1h

Drain and shutdown

The first SIGTERM or SIGINT starts the exit drain (leadership handed off within the budget), then close. A second signal cuts the drain and goes straight to close; a third is the OS default and kills the process. Size the supervisor's kill deadline above the exit-drain budget plus the close guard (the compose stack allows 75 s).

FlagEnvDefaultMeaning
--drain-session-graceNOVAD_DRAIN_SESSION_GRACE30sHow long in-flight work gets to finish after a drain starts: producer sessions before leadership hands off, background jobs before they are cut
--exit-drain-budgetNOVAD_EXIT_DRAIN_BUDGET30sHard bound on the exit drain: the leadership shed a stop signal runs before close
--shutdown-on-stdin-closeNOVAD_SHUTDOWN_ON_STDIN_CLOSEfalseShut down when stdin reaches EOF; a supervisor closes the pipe on its own death, so the daemon never outlives its parent

Logging

Logs are JSON on stderr. Levels follow one rule: Info is lifecycle edges and the periodic star summary, Debug is per-connection and per-job lines, Warn and Error are failures. See observability.

FlagEnvDefaultMeaning
--log-levelNOVAD_LOG_LEVELinfodebug, info, warn, or error; an unparseable level falls back to info
--log-summary-intervalNOVAD_LOG_SUMMARY_INTERVAL1m0sHow often the star summary line (leaders, sessions, append/read MiB/s, backlog age, state flags) is logged; zero disables it

The YAML config file

--config (or NOVAD_CONFIG) names a YAML file whose keys are the flag names without the leading dashes: listen, s3-secret-key, no-audit, auditor-markers-per-era. Durations are Go duration strings (2m, 1h30m), booleans are true/false. A key the file leaves out takes the env or the built-in default; a flag or NOVAD_* variable always overrides the file, which is how one shared file serves a whole fleet while each star sets its own --advertise and --location, and an audit pod overrides the file's no-audit: true with --audit-only --no-audit=false on its own command line.

nova.yaml, a three-zone fleet's shared file
dir: /data
listen: ":7780"
http-listen: ":7788"
metrics-listen: ":9090"

oxia: oxia:6648
oxia-naming: oxia-naming:6648

s3-bucket: nova
s3-endpoint: http://minio:9000
s3-region: us-east-1
s3-path-style: true
s3-access-key: nova
s3-secret-key: novatest            # or NOVAD_S3_SECRET_KEY in the environment
s3-express-buckets: az-1=nova-exp-az1,az-2=nova-exp-az2,az-3=nova-exp-az3

auth: jwt
jwt-hmac-secret-file: /etc/nova/hmac.key
root-credential-file: /etc/nova/root-seed.json

auditor-orphan-grace: 2m
auditor-read-grace: 2m
readplan-ttl: 1m                   # at most half the read grace
auditor-markers-per-era: 256

object-cache-instances: http://objectcache-0:3128,http://objectcache-1:3128
log-level: info

Then per star, on the command line or in its environment:

What differs per star
NOVAD_CONFIG=/etc/nova/nova.yaml NOVAD_LOCATION=az-1 NOVAD_ADVERTISE=novad-1:7780 novad

Note the file above puts the S3 secret in YAML for illustration; the file is a secret-bearing artifact then and needs the permissions to match. Prefer NOVAD_S3_SECRET_KEY, NOVAD_JWT_HMAC_SECRET, NOVAD_ADMIN_PASSWORD, NOVAD_PEER_SERVICE_TOKEN, and NOVAD_ROOT_SECRET from a secret store when the platform offers one.

Source documents

  • cmd/novad/main.go: flag binding and the signal ladder
  • internal/daemon/config.go: Config, defaults, and validation
  • deploy/docker/docker-compose.yml: the compose stack's real flag set

nova CLI reference

Every nova command, subcommand, flag, environment variable, and default, as printed by --help

novagateway reference

Every novagateway flag, its NOVAGATEWAY_* environment variable and default, and how the Gateway sits in front of the fleet

On this page

Config file and dev modeServing and limitsIdentity and placementListeners and TLSAuthMetastore (oxia)Object storage and the Standard write pathExpressAbsorb and the tail windowAuditor and reclamationObject cacheRead plansSettle timingDrain and shutdownLoggingThe YAML config fileSource documents