vibe-live docs

Monitoring

vibe-live exposes Prometheus metrics at /internal/metrics on its internal listener (VIBE_LIVE_INTERNAL_LISTEN_ADDR, :8081 by default). That listener must never be reachable from the internet, so scrape it from inside your network or cluster.

When you cannot reach the internal listener (a hosted instance, or a Prometheus elsewhere), the public port serves the same metrics at /api/v1/admin/metrics, for an admin only:

  1. Signed in as an admin, make an app password under Settings โ†’ App passwords with Read this instance's metrics (admin, for Prometheus).

  2. Give it to Prometheus as a bearer token:

    - job_name: vibe-live
      scheme: https
      metrics_path: /api/v1/admin/metrics
      authorization:
        credentials: vlap_...
      static_configs:
        - targets: [your.instance.example]
    

A Grafana dashboard for an instance is served at /api/v1/dashboards/admin.json and linked under Settings โ†’ Instance โ†’ Monitoring: import it, then pick your Prometheus and the job that scrapes the instance. Its panels: version and uptime, accounts and live channels, the media engine's state, requests by status and response times (the slowest routes too), failed sign-ins and rate-limited requests, the disk and recordings, and deferred tasks. Streamers have one of their own (/api/v1/dashboards/streamer.json).

That password reads metrics and nothing else. It works while its owner is still an admin: a local admin, or someone whose latest sign-in through the identity provider carried the admin group within the last 30 days. App passwords cannot ask the identity provider, so a sign-in without the group ends it at once, and an admin who stops signing in loses it after 30 days. Each replica answers with its own counters, as on the internal listener.

The metrics count things and never name anyone: no user, channel or address appears in a label. Each replica reports its own counters since it started, so sum them across replicas for the instance.

What it reports

Series What it tells you
vibe_live_channels_live, vibe_live_users, vibe_live_uptime_seconds, vibe_live_build{version} the basics: how busy, how big, how long up, which version
vibe_live_tasks_pending, vibe_live_tasks_failed deferred work (deliveries, deletions); failed ones need a look
vibe_live_http_request_duration_seconds{route,code} how long each API route takes to answer, by status class. A histogram, labelled by the route's pattern (GET /api/v1/channels/{id}), never its path. WebSockets are left out.
vibe_live_rate_limited_total{limiter} requests each rate limiter refused: app_password_auth, login_account, second_factor, lookup, mention, force_stop, inbox, federation, chat
vibe_live_rate_limited_by_instance_total{limiter,instance} the same refusals on lookup, inbox and federation, by the instance the requests said they came from: a peer's host, or other (the admin dashboard's Instance menu filters on it)
vibe_live_login_failures_total{reason} failed password sign-ins: unknown_account, bad_password, locked, second_factor. A brute force shows here first; the addresses are in the log.
vibe_live_recordings_bytes, vibe_live_disk_used_bytes, vibe_live_disk_total_bytes what past broadcasts take, and the disk they are on (only where something records)
vibe_live_recordings_limit_bytes or vibe_live_recordings_limit_disk_percent the limit you set in Settings, Recording, in whichever form you set it
vibe_live_engine_status{status} the media engine's health, 1 on its current status: answering, not_answering, in_process (the built-in engine) or unknown

The engine and the media workflows add series of their own: vibe_live_ome_* for OvenMediaEngine, vibe_live_builtin_* for the built-in engine, and the ingest, rendition and thumbnail counters.

Alerts

deploy/prometheus/vibe-live.rules.yml, in the repository, holds alerts on these series. Load it with rule_files: in prometheus.yml, or put its groups in a PrometheusRule if you run the Prometheus Operator.

Alert Fires when
VibeLiveDiskAlmostFull the recordings disk is over 90% full for 10 minutes
VibeLiveRecordingsNearTheirLimit recordings take over 90% of the size you allow
VibeLiveSlowAnswers one answer in twenty takes over a second, for 5 minutes
VibeLiveServerErrors over 5% of answers are server errors, for 5 minutes
VibeLiveSignInFailuresSpike over ten failed sign-ins a minute, for 5 minutes
VibeLiveEngineNotAnswering the media engine stops answering for 2 minutes

The thresholds are starting points: tune them to your instance's normal.

No Prometheus? The app checks the same conditions itself every minute and posts them to a webhook the admin sets in Settings, Instance, Alerts. Use one or the other; both would announce each alert twice.