Streamer autonomy¶
Central is the control plane, not the media path. The streamer processes and serves video independently: it ingests the source, transcodes, records the archive and serves the stream. A central outage therefore affects management, not broadcasting. This page describes what keeps working, what is unavailable and how connections to central recover.
This page covers the case where a streamer cannot reach any central instance. For a single-instance failure while others are running, see Redundancy.
What keeps working¶
- Broadcasting. Streams keep running on streamers: sources, transcoding, archive recording, pushes and video delivery do not depend on central. Direct playback from the streamer continues.
- DVR recording. The archive is recorded without pauses; its window continues growing throughout central's downtime.
- Even a streamer restart. A streamer restarted while central is unreachable starts its streams immediately from the local configuration-slice cache, without waiting for a response from central. It continues retrying synchronization:
WARN sync failed, retrying error sending request for url (.../node-api-v4/sync)
What is unavailable¶
- Management. The console, API, stream list, statistics and sessions — all of central. Configuration changes are not delivered and the layouter does not reassign streams: streamers use their last configuration.
- Playback through central. Media proxying and redirects require central, so new requests to its address get no response. Set a public media address on streamers: central is then needed for the initial redirect, while established connections to streamers survive its failure. Under Cluster → Streamers, open the streamer's settings, go to Connection, set Public media address and save.
- The local streamer. The streamer inside the central process stops with it, as do its assigned streams. This is another reason to run central and broadcasting on separate machines in production.
How connections to central recover¶
The streamer retries sync with exponential backoff capped at about a minute. Once central returns, the connection recovers automatically within that interval: the streamer moves from "Signal lost" to "Online" in the registry, and statistics and sessions become available again. Nothing needs restarting.
Two special cases of central's return:
- Central was restored from a backup with configuration versions older than those applied by the streamer. Central sends an explicit signal; the streamer accepts the full configuration slice as authoritative and continues using it. Configuration version rollback does not disrupt the cluster.
- Central returned with an empty database after a lost disk with no backup. The streamer is no longer registered, and its log reports this explicitly:
ERROR node key rejected; streams keep running from the applied slice, operator action required
Broadcasting continues using the applied configuration slice. To rejoin the cluster, stop the streamer service, clear its state directory (by default /var/lib/optic, where its identity is stored; do not touch the archive on its separate disk), open the join window and join the machine with a fresh token. A new registry record is created, and the layouter assigns streams again. A new token does not replace the stored identity: state must be reset.
What ensures autonomy¶
Autonomous operation relies on the following properties:
- The streamer initiates all communication. Central has no reverse control stream: the streamer needs central to change its configuration, not to run.
- The applied configuration slice is stored on the streamer. Its local cache supports a warm start without central and continued operation with the last configuration during a prolonged outage.
- The group's configuration and registries are stored in PostgreSQL. They are restored from a dump. Server settings and central state directories are saved separately — see Backup and restore.