Stream placement¶

Streams are spread across streamers by the layouter, a central subsystem. You add streamers and create streams, and it decides which machine captures a stream, which keeps a copy of its archive, which serves it to viewers. The decision is recomputed on every change, and when a machine fails its streams move to live ones by themselves.

A layouter decision is called an assignment and ties together a stream, a streamer and a role — what that machine does for the stream. For every assignment central builds the stream's server config and delivers it to the streamer, which is why stream settings are edited in central and not on the machines.

Placement is steered by three settings. Namespaces split the cluster into non-overlapping parts. Labels state requirements and preferences. Limits cap how much a single machine takes.

How streams are spread¶

What the layouter does¶

• Assigns roles and recomputes assignments. A run starts whenever the input changes — a streamer joined, a stream was created, settings were edited. There is no schedule and nothing to launch by hand.
• Accounts for a streamer's capabilities. A stream with GPU encoding is assigned only to a streamer with cards. A stream with an archive needs a streamer with archive disks.
• Watches the load. Among the eligible streamers the less loaded one is picked, by CPU, network and archive usage. An overloaded streamer is relieved by moving streams off it.
• Does not move streams without a reason. While the input has not changed, the assignments stay as they are. A neighbouring machine being underloaded is not a reason to move anything.

Assignment roles¶

Roles are not chosen and not switched on separately. They follow from what is already configured: a stream has a source, so ingest is needed; a stream has a number of archive backup copies set, so copies appear; the cluster has edge streamers, so edge delivery appears. Knowing them is what lets you read the assignments table on the stream's Layout tab.

ingest — the main role, every working stream has it. The streamer receives the stream's source, transcodes it and records the archive when those are configured. A stream has one ingest streamer. This is where the stream's requirements towards a machine are checked, so most rejection reasons belong to this role.

edge — the stream is served to viewers from an edge streamer. The role is assigned to every stream on every eligible edge, and there is nothing to switch on separately. Such a stream captures no source and records no archive, it receives the finished result from the ingest streamer.

dvr backup — the archive is recorded not only on the ingest streamer but on several more. How many copies to keep is set by the DVR backup copies field on the Layout tab.

dvr parts — appears when a stream has been moved to another streamer while recorded archive stayed on the former one. The assignment lets that earlier archive be served from the former machine, so the viewer sees it in the common timeline and moves do not lose archive.

Failover¶

The streamer states are described in the registry, and the layouter reacts to them as follows.

• Signal lost. The assignments freeze — nothing is removed, nothing is created, and the machine receives no new streams. A short outage does not move streams, and once the link is back the machine keeps running the same ones.
• Offline. The streams move to live machines as soon as suitable ones are found.

Edge delivery is the exception. It survives any connection state, because an edge brings the stream up when a viewer asks for it, and there is no point removing the assignment over silence. A streamer taken out of the cluster loses all its assignments.

How to steer placement¶

The streamer's settings live on the Cluster tab of its page.

A stream has a namespace and labels of its own, set on the Layout tab of its page — the same place its assignments are shown.

Namespaces¶

A namespace is a hard boundary. A stream is assigned only to streamers with the same namespace, and there are no exceptions across namespaces. An empty value is a namespace too, so a stream without one is assigned only to streamers without one.

This is how a cluster is split into non-overlapping parts — by tenant or by site, for example. A streamer's namespace is set on the Cluster tab, a stream's on the Layout tab.

Labels¶

A label is an arbitrary mark you put on a streamer and on a stream. What it means is up to you: the layouter only compares labels when it picks a machine. This is how a condition the system cannot know about enters placement. A label can forbid placement the way a namespace does, but it acts more flexibly and within a single namespace, and the condition is set per stream.

A streamer's labels are set on the Cluster tab, a stream's on the Layout tab. The way a label is written decides what it means. The word fast below is just an example, any word of yours can take its place.

• fast — a property. This is how a machine or a stream is marked without requiring anything.
• required_fast — a requirement. A stream carrying it is assigned only to a streamer that also has the label fast. The rule works both ways — a streamer with required_fast takes only streams with that label.
• preferred_fast — a preference. All else being equal a streamer with the label fast wins, but its absence does not stop the assignment.

What is compared is the word itself, not the way it is written. A required_fast requirement is satisfied by any of the three forms on the other side — fast, required_fast or preferred_fast. A preferred_fast preference counts the same way.

Limits and load¶

A streamer's load is measured for the machine as a whole — all of its CPU and all of its traffic, including delivery to viewers, transfers between streamers and unrelated processes. Assignment roles do not affect the measurement.

Three fields on the Cluster tab set the caps.

• Stream limit — how many streams the machine may carry. Zero keeps it in the cluster but empty.
• Network limit, kbit/s — the machine's traffic cap. The layouter adds no streams beyond it, and relieves the machine when it is exceeded.
• DVR usage limit, % — the archive disk usage at which the streamer counts as full.

The caps restrict ingest and archive backup assignments. Edge delivery and dvr parts are not restricted by them — those roles are assigned regardless of the machine's load.

What is next¶

What the layouter decided and why a stream ended up where it did is shown by Placement status.