Skip to content

Cluster DVR

The task of archive storage in a distributed video delivery environment creates several problems:

  • It's necessary to ensure the safety and availability of an archive. RAID-systems do not solve the problem because when you turn off the server all data on it becomes unavailable;
  • For a popular content it's necessary to ensure the delivery of the archive closer to users and to ensure load reduction on source servers;
  • In a geo-distributed video delivery environment you must be able to restore the integrity of the archive on the secondary servers after a loss of the source server.

The easiest option for clustered DVR access is to enable DVR on the source and on the secondary servers.

Clustering of DVR in Flussonic can be turned on easily: use the source directive to receive the video:


cluster_key abcd;
source streamer:8081 {
  dvr /storage 2d;
}

In this case, the local server, even if DVR is turned off, will respond to the DVR requests (but not via any protocol), receiving the missing video from the source.

An important feature of Flussonic is the ability to use separate segment cache on the SSD for reducing load. Usually up to 90% of all visits is for the last day of a video, so for broadcasting of a large-scale events you can use SSD to reduce HDD load:


stream example {
   url fake://fake;
   dvr /storage 80d;
   cache /storage/cache 2d 800G;
}

When using DVR clustering, you can omit /storage on the secondary servers and specify only a segment cache:


cluster_key abcd;
source streamer:8081 {
   cache /storage/cache 2d 400G;
}

In this configuration the secondary server will use a segment cache to store DVR, but actually the entire archive will be controlled by the source server. In case of loss of communication with the source, the secondary server will not be able to respond to requests to the archive.

Merging the DVR archive from different servers

When working in a cluster or with several stream sources, Flussonic merges the DVR archive requested from the secondary server (restreamer or edge server) with the main archive (origin or source server). The goal is to keep the DVR available when the primary source fails and switches to a backup source.

If you have multiple M4F stream sources and a cluster with multiple levels in the chain of servers (for example, DVR - Cache - Edge), you may need to configure which servers the archive should be requested from. This section tells about this.

We will consider a cluster containing:

  • a source server that ingests a stream and writes an archive
  • at least two levels in the chain of edge servers, and at each level there can be more than one edge server.

The stream's DVR archive is recorded and stored on a source server, and edge servers are used only to deliver a stream to end-users, for ensuring increased throughput. We will consider the architecture where there are two levels in the chain of edge servers.

By default, Flussonic looks for DVR data on all configured stream sources, but without chaining (that is, connecting to further levels of Edge servers). If a stream has a number of M4F sources, Flussonic will form the archive using all these sources, but it will take into account only the first level of edge servers (or, if the source server has other sources, the archive is not downloaded from that other sources). This behavior was introduced in Flussonic 21.01.

However, you can control how the archive should be transmitted over the chain of servers, with the option remote_dvr added after the source URL:

  • remote_dvr=nochain — no requests to DVR further down the chain, only the first level of edge servers are used to obtain the archive (applied by default).
  • remote_dvr=chain — requests to DVR with recursion to source sources.
  • remote_dvr=none — no requests to sources except the currently used source (this helps save resources).

Accessing servers in a chain can lead to loops, so use this feature with caution.