Live stream sources and LSI¶

media.live.inputs

LSI (Live Source Input) manages the inputs of a live stream. It ingests media from the selected source, monitors its condition, and handles recovery when problems occur. If the stream has backup inputs, LSI assesses their suitability and chooses where to resume ingest.

Opening an input address once is not enough for round-the-clock broadcasting. A source may keep its connection open but stop delivering video. The primary route may recover for only a few seconds. A backup may be unavailable just when it is needed. LSI monitors sources throughout the stream's lifetime and makes decisions based on media data.

Source management is built into the Sapsan pipeline:

• Ingest monitoring detects missing frames and problems with individual tracks.
• Quality assessment helps detect stream corruption before ingest stops completely.
• Backup probing provides information about backup availability before the primary input fails.
• Coordinated switching prepares a new source and passes its media further down the pipeline.
• Observability shows the selected input and the reasons behind decisions, and lets you measure time spent on backup inputs.

What makes a source working¶

A successful network connection does not yet mean that a source is ready for broadcast. LSI considers incoming media data and whether there is a point from which processing can start. For a frame-based video input, this is a keyframe; for an audio-only source, an audio frame. When ingest preserves existing fragments, a fragment boundary is used.

A source's condition depends on the observations available for its protocol and the configured rules:

LSI distinguishes failure from quality degradation. After a failure, ingest from that input cannot continue: for example, the no-frame timeout or the configured video timeout has expired. The input is closed, and a search for an alternative begins.

With quality degradation, media continues to arrive. For example, audio disappears or the number of continuity errors exceeds the allowed threshold. LSI looks for a more suitable source while maintaining the current ingest. If no healthy backup is available, it continues using the degraded source: the remaining broadcast content stays available.

This monitors the technical condition of the media. Frames that arrive on time but show a still image do not, by themselves, constitute a video timeout. Degradation rules are covered under source quality monitoring.

How LSI selects a source¶

Multiple entries in inputs describe alternative sources for one stream. During normal operation, media from the selected input is passed down the pipeline. By default, priority follows the list order; an explicit source_options.priority overrides it. A smaller number means higher priority.

A clean window is an interval without violations of the quality rules. Its duration is set by return_alive_secs, with a default of 15 seconds. When returning to a higher-priority source, a successful connection alone is insufficient: LSI must observe stable ingest. This protects the broadcast from repeated switches to a source that recovers only briefly.

Emergency failover after the current input fails does not wait for this full window: ingest resumes as soon as the candidate is ready. If the backup still needs to be opened, recovery time includes establishing the connection and waiting for a suitable starting point.

Failed connections are retried with an increasing delay. An unavailable address therefore does not cause a continuous loop of immediate attempts. Detailed parameters are described under source failover.

How backups are probed¶

LSI can probe a backup while the primary source continues working. The check receives real media data and assesses its quality. The result includes the source's condition and observation time; it is available through the API along with track information.

Periodic checks are configured with recheck_secondary_inputs_interval: the default is 900 seconds, and 0 disables them. This is a scheduling interval for background checks, not a promise to open every backup exactly once every 900 seconds. The current work of LSI and reconnection delays affect when a check takes place.

A successful background check ends by closing the backup input. A working state on an unselected source therefore means it was healthy at the last observation. Use checked_at to assess how fresh the result is. Continuous parallel ingest from all backups is not implied.

The checking method depends on the input. A source that Sapsan connects to can be opened temporarily. With publishing, the sender initiates the connection: LSI observes the incoming publication but cannot open it on the sender's behalf. The clean return window described above applies to inputs that can be probed; an accepted incoming publication may be selected on its first suitable starting frame.

Switching and broadcast continuity¶

For a prepared switch, LSI maintains the current ingest until the replacement is ready. For example, a stream is using a backup input and the primary has recovered:

1. LSI opens the primary input; consumers continue receiving media from the backup.
2. The primary passes the clean window.
3. At a suitable keyframe or fragment boundary, LSI selects the primary input.
4. The pipeline receives a source-change notification and new media parameters before frames from the new input.

Waiting for the primary source to be checked therefore does not turn into a broadcast pause. Switching takes place within the same named stream; the operator does not need to stop and recreate it.

Timestamp processing also accounts for source changes. For inputs that need alignment to server time, the relevant pipeline component establishes this alignment again. Protocols with a trusted calendar timeline and fragment-preserving modes follow their own rules for retaining timestamps.

These mechanisms make seamless switching possible with prepared, compatible sources. The viewer's experience depends on the entire chain:

• Media compatibility. Align codecs and track composition. Also check video and audio parameters: a format change may require decoder reconfiguration.
• Content synchronization. Two sources of the same TV channel may have different delays. Without content alignment, switching may repeat or skip part of the programme.
• Availability of a starting point. A long GOP increases the wait for a keyframe. Inputs with existing fragments switch at a fragment boundary.
• Type of failure. If the current source suddenly disappears, switching alone cannot recover frames already lost. The detection timeout and backup readiness affect the pause.
• Output pipeline. Test switching with actual delivery protocols and players, and, when transcoding, with the selected transcoder configuration.

LSI does not check complete equivalence of encoding parameters before every switch. Having two addresses in inputs alone does not guarantee an imperceptible transition. Prepare the sources and verify end-to-end playback when designing the system. If all inputs are unavailable, the separate slate feature can play a prepared file.

Configuration example¶

Sapsan receives configuration through the central API. The following YAML fragment defines two alternative UDP sources. The lsi settings define the stream's overall policy, while source_options refines supported parameters for an individual input.

streams:
  - name: tv1
    inputs:
      - udp:
          host: 239.0.0.1
          port: 5000
        source_options:
          priority: 1
      - udp:
          host: 239.0.0.2
          port: 5000
        source_options:
          priority: 2
    lsi:
      source_timeout: 5
      video_timeout: 3
      audio_timeout: 3
      cc_errors_limit: 10
      return_alive_secs: 15
      recheck_secondary_inputs_interval: 900
      reconnect_initial_delay: 1500
      reconnect_delay_step: 5000
      reconnect_max_delay: 60000

In this example, receiving no frames for 5 seconds or no video for 3 seconds constitutes an input failure. Receiving no audio for 3 seconds constitutes quality degradation. The continuity error limit is 10 per assessment window; exceeding it degrades the source's condition. A 15-second clean window is required to verify return to the primary.

The timeouts and observation intervals in this example are in seconds. The reconnect_* parameters are in milliseconds: an initial delay of 1500 ms, an increment of 5000 ms, and a limit of 60000 ms. These reconnection parameters are set in lsi.

The timeout values here are examples. Choose them according to source behavior and the acceptable broadcast interruption. By default, source_timeout is 60 seconds, and separate video and audio timeouts are disabled.

What to monitor¶

The central API shows the condition of a stream's inputs:

curl -fsS -u "$ADMIN_USER:$ADMIN_PASSWORD" \
  http://central:8080/central/api-v4/streams/stats/tv1 \
  | jq '{name, inputs: [.inputs[] | {
      url, current, status,
      problems: (.problems // []),
      checked_at, error, failure_cause
    }]}'

The current field identifies the selected input, while status describes its condition. These are independent: the current input may be degraded, while a backup is working. During an emergency search, current does not yet confirm that ingest has resumed: consider status and actual playback as well.

• unchecked — no observation result is available yet.
• starting — the input is open, but a suitable starting point has not yet arrived.
• working — the source is healthy according to the available observations.
• degraded — media is arriving with quality rule violations.
• dead — the source is unavailable or has been declared failed.

problems reports detected violations, while error and failure_cause help diagnose errors. checked_at contains the observation time in UTC, formatted as RFC 3339; it may be absent for a source that has not yet been checked. Requesting statistics does not itself initiate a backup check.

The following signals are useful for operational monitoring:

• Current input degradation — current: true together with degraded or a failure.
• Loss of backup availability — a backup input has failed or has had no fresh confirmation of health for a long time, while current delivery still works.
• Frequent switching — input_switches increases; check source stability and return settings.
• Extended operation on backup — num_sec_on_secondary_input increases; primary recovery needs attention.
• No media — num_sec_no_data increases.

Detailed .input counters are available by requesting statistics for one stream from the streamer performing ingest: /streamer/api-v4/streams/stats/tv1. The compact state in central does not replace this section. A request example and interval calculation rules are provided under measuring time on inputs.

input_switches counts transitions between different configured inputs; reconnecting to the same input is not itself a switch. Time counters distinguish the first input in the list from the others, even if priorities are overridden. These are cumulative values: compare successive observations to assess a period, and account for statistics resetting on restart.

How to verify failover before going live¶

1. Confirm that the expected primary input is selected. Obtain a fresh backup check result and compare inputs[].media_info: input parameters must match the system design.
2. Interrupt the primary source in a controlled way. Check the change in current and increase in input_switches while observing actual playback.
3. Restore the primary source. For inputs that Sapsan connects to, confirm that the backup continues working during the check and that return occurs after the clean window.
4. Separately test quality degradation without disconnecting, for example loss of the audio track with audio_timeout enabled.
5. Test backup failure while the primary is working: monitoring should detect the loss of backup availability before the primary input fails.

What LSI provides for a broadcast service¶

FFmpeg provides media processing tools and can reconnect — for example, HTTP has reconnection options. When building a service around an FFmpeg command, managing the lifecycle of multiple sources requires a separate solution.

In Sapsan, LSI brings this work together. The operator defines sources and policy; the streamer monitors quality and prepares switches, while the API supplies state for monitoring. Failover thus becomes a managed part of round-the-clock broadcasting: you can check a backup route in advance, explain why the source changed, and confirm that the primary has recovered consistently.