Wall-clock alignment¶

media.source.realtime_aligner

For a live stream, knowing frame order and playback rate is not enough. The streamer needs to know which calendar time each frame corresponds to: it can then record it in DVR and later find the required moment by date and time.

Sources provide different information for this decision. Some have only a relative timeline; others have their own clocks, which cannot always be trusted. The network may deliver media with evenly spaced timestamps in bursts. Sapsan maps these timelines to wall-clock time while preserving the intervals between frames, and reassesses the mapping when source behavior changes.

Media time, source clock, and arrival time¶

A frame has timestamps: DTS specifies its decoding time, and PTS specifies its presentation time. They determine its position on the media timeline, but do not necessarily contain a date and time of day.

This information may become available at different times within one stream. For example, an RTSP input first receives frames with relative RTP timestamps, then obtains an absolute time mapping from RTCP. Reconnecting a camera or switching sources may change the timeline basis again.

MPEG-TS without calendar time¶

MPEG-TS timestamps allow video and audio to play at the correct rate, but do not say that a frame belongs to, for example, 14:00:00 today. The counter may start at an arbitrary value and wrap around its range.

Sapsan chooses an offset between the media timeline and the streamer clock. It then applies that offset to DTS and PTS, preserving the distances between timestamps:

calendar_time = source_time + offset

For example, a frame with relative DTS 42.000 s is anchored to 14:00:00. The next frame, with DTS 42.040 s, receives the time 14:00:00.040. Its timestamp is not replaced with the time of the next socket read.

The archive thus gets a calendar timeline based on the stream's arrival in Sapsan. For a source without absolute timestamps, this is a server-side anchor: it cannot recover the unknown original capture time or delay before reaching the streamer. With an input buffer enabled, anchoring happens after the buffer releases frames, so its delay also affects the timeline's position.

RTSP and WebRTC: absolute time needs validation¶

RTP-based inputs, including RTSP and WebRTC, can obtain absolute time through RTCP Sender Reports. But a calendar timestamp does not guarantee a correct clock: a camera may have lost its time settings, run without synchronization, or report a time far in the past or future.

In the normal alignment path, Sapsan distinguishes calendar timestamps from relative ones. Unix timestamps no earlier than the year 2000 are classified as calendar time. If such a timestamp differs from the streamer clock by no more than 60 seconds, it is preserved: delivery timing does not become a reason to assign the source a new time.

If the difference exceeds the trust window, the timestamp is treated as a timeline requiring server-side anchoring. The check is symmetric for past and future. This prevents current broadcasts from being recorded under an obviously incorrect camera calendar.

The trust window does not guarantee precise synchronization: clock errors within it may persist. Source and server time synchronization remains necessary to match an archive recording accurately to a real event.

The decision also applies to individual tracks. A track with trusted calendar time does not receive the offset chosen for a relative timeline again: otherwise, the server could add the Unix epoch to a timestamp that is already absolute.

Burst delivery: HLS → HTTP MPEG-TS¶

An intermediate server may receive several seconds of HLS, parse a segment, and send its content over HTTP MPEG-TS in a single burst. A pause follows until the next segment. Media time advances evenly within each burst, while frame arrival time varies sharply.

If every frame were timestamped by its arrival time, several seconds of video would collapse into a short network burst, and the pause between segments would become a timeline gap. Continually changing the offset with every fluctuation would introduce repeated discontinuities and re-anchoring.

Sapsan processes this input in sequence:

1. The input buffer collects frames and releases them at the DTS rate. Timing within the stream determines release intervals, while the queue absorbs network bursts.
2. The aligner maps the resulting media timeline to calendar time. It retains the selected offset while the difference stays within tolerance.
3. Downstream components receive anchored timestamps for transcoding, packaging, and recording.

For a tshttp input, the default buffer depth is 30 000 ms. Configuration is supplied through the central API:

streams:
  - name: tv1
    inputs:
      - tshttp:
          url: http://origin.example/live/tv1.ts
        source_options:
          buffer_ms: 30000

buffer_ms is set in the specific input's source_options. It is a target depth in media time, not a byte count. An explicit 0 disables the buffer; a positive value also enables it for other input types. It is disabled by default for other types. Increasing the depth covers longer pauses at the cost of latency and memory.

The buffer cannot compensate for an indefinite source outage. Once queued frames run out, output pauses. If too much material accumulates, older content may be discarded to reduce lag behind live; these events are visible in statistics.

When anchoring changes¶

For a timeline requiring a server offset, the aligner checks the difference between the current clock and the already shifted DTS. If the absolute difference becomes greater than 5 seconds, a new offset is chosen. This responds to timestamp resets, accumulated rate drift, and timeline changes after reconnection.

The decision uses one leading audio or video track. Video takes priority over audio. Other tracks with relative time receive the common offset: their differences must not cause them to alternately rebuild the entire stream timeline. If the leading track has not arrived for more than 5 seconds, another arriving audio or video track may take the lead.

The offset applies to both DTS and PTS. The aligner preserves their relative position; it does not repair existing audio/video desynchronization within the source.

Re-anchoring sends a timeline change event into the media pipeline. Downstream packaging may close current fragments and start new ones. Re-anchoring can move time forward or backward: the mechanism restores the wall-clock mapping rather than promising to hide every source jump without a discontinuity.

Sources with an already prepared calendar timeline use a separate time-preserving mode. Direct HLS and DASH inputs pass through without this correction; their normal lag of several segments does not cause continuous re-anchoring. Direct HLS ingest and ingest of HTTP MPEG-TS produced from HLS by an intermediate server therefore require different behavior. Time preservation also applies to internal paths carrying already anchored frames, to avoid changing the timeline again at every stage.

Alignment is built into the live pipeline. The 5-second and 60-second thresholds describe the current implementation; no separate YAML settings for them are exposed in stream configuration.

What to monitor¶

Request statistics from the streamer ingesting the source:

curl -fsS -u "$ADMIN_USER:$ADMIN_PASSWORD" \
  http://streamer:8080/streamer/api-v4/streams/stats/tv1 \
  | jq '{name, dts_source, source_timing_offset_ms,
         source_timing_offset_min_ms, source_timing_offset_max_ms,
         source_timing_jitter_ms, source_timing_input,
         source_timing_window_started_at, input}'

The dts_source field identifies the timeline basis:

• realtime — time is maintained through an offset to the streamer clock.
• input — source timestamps are preserved: they passed calendar time validation, or the input uses a time-preserving mode.

The field may be absent before frames arrive and after prolonged silence. In metric export, stream_dts_source uses 1 for realtime and 2 for input.

source_timing_offset_ms shows the current offset measured by the timing monitor. The bounds source_timing_offset_min_ms / source_timing_offset_max_ms and source_timing_jitter_ms refer to the completed one-minute window; source_timing_input and source_timing_window_started_at identify the source and window start. For a relative timeline, the monitor chooses its own reference point, so these values do not measure the unknown delay between the camera and streamer.

The monitor runs before the aligner but after the input buffer. With buffering enabled, its measurements already reflect smoothed delivery. To assess the original unevenness, compare them with buffer state and underruns rather than relying on a single jitter graph.

The input section provides buffer measurements:

Zero-valued fields may be omitted from the response. Compare counter increases over an interval; a missing statistics section does not mean that a check completed without errors.

Three scenarios are useful to verify in operation:

• Steady relative stream — the archive timeline follows the streamer clock while frame intervals are preserved.
• Source with an incorrect clock — an obviously untrustworthy calendar is replaced with server-side anchoring; after its clock recovers, the source may supply its own time again.
• Stream arriving in bursts — the buffer smooths output; long pauses and overflows are visible in statistics rather than hidden by constantly reassigning each frame's timestamp.