All docs pages ↓
Metrics
GET /metrics is a Prometheus text exposition of the whole service. A scrape always carries a credential: the root token, or a read-only metrics token that opens this route and nothing else.
Commands are written with the odu shorthand.
Scrape it
Mint a metrics token, then scrape with it:
odu surface metrics token_mint --input '{"rootToken":"'"$ODU_ROOT_TOKEN"'","label":"prometheus-pod"}' --json
curl -H "Authorization: Bearer $METRICS_TOKEN" https://cache.example.com/metrics
A metrics token keeps the root token, which administers the server, out of Prometheus.
Metrics tokens
| API verb | What it does |
|---|---|
metrics_token_mint |
Mints a metrics token. The plaintext is returned once. |
metrics_token_list |
Lists every metrics token, and when each was last used. |
metrics_token_revoke |
Ends one at once, with no restart. |
- All three take the root token. A service started without one serves no metrics at all.
- Only a sha256 of each metrics token is kept, in
<state>/metrics/tokens.json, readable by the owner only.
What is measured
Each part of the service contributes its own numbers to one exposition.
| Metric | Type | Labels | Counts |
|---|---|---|---|
odu_cache_requests_total |
counter | bucket, kind (narinfo|nar), outcome (hit|miss|redirect|refused) |
requests the cache served |
odu_cache_bytes_total |
counter | bucket, direction (served|uploaded), mode |
NAR bytes served and uploaded |
odu_cache_uploads_total |
counter | bucket, mode, outcome (recorded|failed) |
pushes and imports |
odu_cache_gc_evicted_total, odu_cache_gc_runs_total |
counter | bucket / — |
collections |
odu_cache_hits |
gauge | bucket |
SUM(hits) over the paths a bucket holds now. A collection lowers it, so it is not a _total. |
odu_cache_paths, odu_cache_bytes, odu_cache_pins |
gauge | bucket |
read from the database at the scrape |
odu_cache_last_gc_timestamp_seconds |
gauge | — | the last collection since the service started, 0 when there has been none |
odu_runs_total |
counter | outcome |
runs in the history on disk, each counted once |
odu_run_duration_seconds |
histogram | outcome |
run verdicts, in buckets from 10s to 2h |
odu_node_duration_seconds |
histogram | recipe, platform, outcome |
each node_status entry in a run’s journal, counted once |
odu_lane_reachable, odu_lane_held |
gauge | host, platform / host |
the last probe of the machines. Absent until something has probed. |
odu_metrics_contributor_up |
gauge | contributor |
0 when a part of the service failed this scrape. The scrape is still served. |
odu_unhandled_total |
counter | kind (rejection|exception) |
errors the service caught and kept serving through. Both labels are present from the first scrape. |
Counters and restarts
- Counters belong to the process and reset when the service restarts, which
rate()handles. Numbers that must survive a restart are gauges read from storage. - A scrape reads at most the 200 newest runs, at most every ten seconds, so it costs the same on a history of ten thousand runs as on two hundred.
- Each run and each journal entry is added once to totals that only grow. A counter read off a sliding window would fall each time a run left it, and a scraper reads a fall as a reset.
- The run totals therefore start from what those 200 runs held when the service started.