Skip to content

Flussonic UGC implementation guideline

This document explains how to design and implement a UGC service or a platform with Flussonic Media Server. Flussonic Media Server is a multi-purpose software solution for launching a high-load video streaming service of any scale.
This document is intended for:

  • Software architects
  • CTOs
  • Product or project managers of the companies

of the companies that want to develop their own UGC streaming platform.
This document doesn't include information about installing or configuring the Flussonic Media Server. For information on installing Flussonic Media Server and getting started with it, see Installation and Quick Start.

Introduction focuses on the objectives that Flussonic Media Server can solve to create a UGC streaming platform and where UGC streaming platforms are used. The architecture of a typical UGC streaming platform describes the architecture design of a UGC platform or a service, its key components, workflow, and ways to make your system resilient and scalable. Analyze and establish requirements examines requirements for the service. Design the network architecture shows how to estimate the network bandwidth for the service and what you need to create an architecture design diagram. Example of an implementation strategy for a UGC service describes a real-life project.

Table of contents

  1. Introduction

  2. The architecture of a typical UGC streaming platform

    2.1. Key components
    2.1.1. Publishing server
    2.1.2. Transcoding server
    2.1.3. DVR server
    2.1.4. Egress server
    2.1.5. Central
    2.2. Workflow
    2.3. Redundancy and scaling strategies
    2.3.1. Cluster of publishing servers and DNS balancing
    2.3.2. Load-balanced transcoder cluster
    2.3.3. DVR cluster with replication
    2.3.4. CDN

  3. Analyze and establish requirements

    3.1. Define a niche
    3.2 Study your target audience
    3.3. Define the content and its creators
    3.4. Define the key features

  4. Design the network architecture

  5. Example of an implementation strategy for a UGC service

    5.1. Analyze and establish requirements
    5.2. Design the network architecture

  6. Glossary

Introduction

UGC is both:

  • Streams with ultra-low latency within one second, such as group conversations in Google Meets or Zoom
  • Streams in the social networks with a latency of several seconds

Streaming UGC platform is used for streaming the content like video games, business events or webinars.

UGC streaming platforms are used in the following fields:

  • Gaming. Live streaming of tournaments and competitions in cybersports, walkthroughs and reviews of video games.
  • Sports. Live streaming and recording of tournaments and competitions in various kinds of sports.
  • Education. Live streaming and recording of lectures, seminars, webinars, courses and specializations.
  • Religion. Live streaming and recording of worship services.
  • Events. Presentations of new mobile phone models, streaming of cultural events.

The project starts with defining the goals and objectives.

Project goal is the result we want to achieve that adds value to the business; the purpose of the project. For example, to increase sales revenue or reduce company expenses.
Project objectives are specific steps or tasks to pursue the goal.

Flussonic Media Server allows you to solve the following objectives when creating a UGC service or a platform:

  • Receiving streams from creators with the highest possible quality and stability.
  • Ensuring the hardware is used under uneven load during the time of day.
  • Ensuring even distribution of network traffic among servers.
  • Charging creators for retransmitting and storing the recordings.
  • Recording, storing, and playing back the recordings from the archive to viewers.
  • Multistreaming content to social networks.
  • Providing a comfortable environment for online video and audio conferences without the need to install any additional applications.
  • Charging viewers and paying royalties to creators.

Without establishing goals and objectives, you can not properly specify the requirements for the project and build a diagram of the service architecture.

The architecture of a typical UGC streaming platform

This chapter covers the following topics:

The following diagram shows an architecture of a typical UGC streaming platform built with Flussonic Media Server:

Diagram 1. UGC platform architecture design

UGC platform architecture design

Here:

Key components

Here are the key components that make up a UGC streaming platform:

  • Publishing server is a server that receives a stream from the creator.
  • Transcoding server is a server that transcodes a stream into an array of streams at different resolutions and bitrates. Optional component.
  • DVR server is a server that takes incoming video streams, records, and stores them, then sends them on demand. Optional component.
  • Egress server is a server that delivers the content to the viewers.

  • Central ia a management server in Flussonic that provides a central point of access for managing server configurations for every server.

Publishing server

Publishing server allows to solve the following objectives:

  • Receiving streams from creators with the highest possible quality and stability.
  • Multistreaming content to social networks.

Flussonic receives live streams over various protocols:

  • WebRTC

    • Requires no additional software installation to start streaming. A creators only needs a browser and an Internet connection.
    • Allows to automatically adjust the video quality to the speed of the Internet connection using the adaptive bitrate (ABR) algorithm. The higher the bitrate, the better the audio and video quality for the creator.

  • SRT

    • Provides low-latency and uninterrupted publishing by reducing video quality.

  • RTMP

    • Allows creators to use almost any video editing equipment and open source solutions.

Publishing server repackages the stream to the Flussonic-to-Flussonic streaming protocol M4S* and then transports it to one of the servers:

  • the transcoding server
  • the DVR server
  • the egress server

depending on the architecture design and the objectives.

Note

M4S is a real-time streaming protocol for transmitting the video only between Flussonic servers. The protocol supports all Flussonic codecs. To learn more about M4S, refer to M4F and M4S protocols.

Read also:

Transcoding server

The transcoding server prepares the content to deliver it at the best possible quality to viewers:

The transcoding server receives the stream and transcodes it, getting a multi-bitrate stream at the output.

Creating multi-bitrate stream

To prepare the content, prepare the multi-bitrate streams—synchronized video tracks with different resolutions and bitrates. For the transcoder to prepare the stream for all viewers, the input stream should be at the highest bitrate settings.

Here are the optimal video profiles for the video stream delivery:

  • 1920x1080p
  • 1280x720p
  • 896x504p

Also client device determines the profile for video delivery. For example, for the majority of smartphones 896x504p profile is enough, however, TVs require at least 1920x1080p. The transcoder can both downscale and upscale the resolution of the stream to deliver it to viewers. For example, a creator publishes a stream at 720p resolution, and the transcoder upscales the resolution to 1080p to deliver it to viewers.

Note

The higher the stream resolution and bitrate, the greater the latency.

If you receive the streams from various sources simultaneously, such as a browser, a webcam, OBS, or a video encoder, transcode each such WebRTC, RTMP, and SRT stream, rather than repackage it. WebRTC, RTMP, and SRT have different and not fully compatible codecs.

The transcoder also applies the following delays:

  • Three to ten seconds to achieve the highest quality at minimum bitrate.
  • One to two seconds to achieve acceptable quality with widely varying bitrates.
Load balancing of the transcoding servers

To distribute the load between the transcoding servers and prevent the service from stopping, use the external stream management with the config_external mechanism. This way you don't need to configure the streams manually.

Monitoring of the volume of the transcoding traffic

The volume of the transcoding traffic is the sum of bitrates of tracks in the stream profile. With this value, you can determine the cost of transcoding for each creator.

Based on the source stream bitrate, it can be a combination of profiles. For example, if the 1920x1080p source stream is at 5 Mbps bitrate, then recommended profiles are 1920x1080p at 4 Mbps, 1280x720p at 2 Mbps, and 896x504p at 900 Kbps. Then the volumes of the transcoding traffic are:

  • For 1920x1080p: 4 000 Kbps + 192 Kbps = 4192 Kbps
  • For 1280x720p: 2 000 Kbps + 128 Kbps = 2128 Kbps
  • For 896x504p: 900 Kbps + 96 Kbps = 996 Kbps

Transcoding can be done with a specialized dedicated hardware, a CPU or an external or integrated GPU. Flussonic Media Server has a built-in transcoder. It supports transcoding with a GPU or a CPU. To transcode streams, you can use media server or Flussonic Coder.

Transcoding is a computationally expensive process, and it puts a heavy workload on the CPU and GPU. Changing video codec, resolution, or bitrate requires a high amount of system resources, unlike transmuxing. Transcoder isn't required for transmuxing.

Transcoding and transmuxing operate on different levels: transcoding—content or data, transmuxing—container and streaming protocol.

Note

If you need an expert advice and assistance when to choose the right equipment for your project, contact our technical support team at support@flussonic.com.

For more information about transcoding and supported codecs and protocols, see:

DVR server

DVR server records live streams and stores the recordings of them in an archive.

DVR server provides your viewers an opportunity to pause, rewind, fast-forward, re-watch, and record any live stream and watch it later.

Flussonic Media Server provides features to work with the archive, such as:

Flussonic Media Server can work with cloud storages. It can upload files to the cloud storage and download them from it.

Flussonic Media Server also allows to export archive fragments as MP4 files.
For more information, see Export DVR segment to MP4 file and Download the DVR segment to MP4 or MPEG-TS file to a local computer.

To reduce the load on the storage and speed up the distribution of VOD content, set up SSD caching.

For more information about DVR and its features, see DVR.

Egress server

The egress server allows you to accomplish the following objectives:

  • Provide a comfortable environment for people to communicate on the Internet in real-time video and audio conferences.
  • Deliver the recordings of streams to the viewers.

Latency in video or audio is critical to a real-time live stream. Viewers may experience issues with audio and video, leaving them dissatisfied with the quality of the service. To provide a comfortable environment for real-time video and audio communication online, use WebRTC. WebRTC allows you to :

  • Create a sense of live communication between the participants with latency less than a second.
  • Automatically adjust the video quality to the speed of the Internet connection using the algorithm of adaptive bitrate (ABR). This makes it possible to expand the audience of private chats and for viewers with an unstable or slow Internet connection to watch the live stream.
  • Watch the live stream on any device.
  • No additional software required. All a viewer needs is a web browser and connection to the Internet.

Live streams with a delay that is enough to respond to the chat, but not enough to communicate with viewers live are most common. In such cases, a small latency isn't critical, so use HLS, DASH, and MSS protocols to deliver streams to viewers. It makes it possible to:

  • Watch live streams on any device.
  • Expand the audience that doesn't require ultra-low latency.
  • Improve the effectiveness of the sales funnel.

Egress server captures the streams from one the following servers:

  • the transcoding server
  • the DVR server
  • the publishing server

Egress server then delivers the stream to the viewers directly or with the CDN, depending on the objectives and the architecture design.

For more information, see Pushing a stream to other servers.

To show the streams to the viewers you can use Flussonic web player and embed it on your website. To integrate the player with your website, use the Flussonic Streaming API.

Central

Central helps to:

  • Ensure the hardware is used under uneven load during the time of day.
  • Ensure even distribution of network traffic among servers.

Central acts as a management server in Flussonic that provides a central point of access for the following tasks:

  • Managing and coordinating server configurations for all the servers in a content delivery pipeline.
  • Authentication and authorization of creators and viewers.

Central allows you to make a highly available server configurations for the servers in the video delivery pipeline. This means you don't have to configure each server manually. Central interacts with other servers using a built-in mechanism—config_external.

Central also handles load balancing between servers in the cluster, redirecting requests to the least loaded servers.

Flussonic provides the Flussonic Central API to communicate with Central.

You can implement the authorization using Flussonic built-in tools or with an external backend.

If you don't have a backend for authorizing creators and viewers or don't want to create one, you can use the Flussonic built-in tools, such as:

If you already have an authorization backend, configure Flussonic to access it using these tools:

Flussonic also provides the Flussonic Authorization Backend API to work with the authorization backend.

Find the information about the components and their features in the Table 1:

Table 1. Content delivery pipeline components and their features

Component Features
Publishing server - receiving streams
- multistreaming to other streaming platforms and socials
Transcoding server - changing audio and video parameters (codec, bitrate, resolution, etc.)
- creating a multi-bitrate stream
- overlaying logo
DVR server - recording and storing copies of incoming streams
- working with the archive and its features (live-to-VOD, timeshift, etc.)
- playing streams from the archive (rewind, watching the current live stream from the beginning)
Egress server delivering streams to viewers: transcoded streams (for live viewers) and copies of transcoded streams (for DVR users)
Central - managing and coordinating server configurations for all the servers in a content delivery pipeline
- authentication and authorization of creators and viewers
- server load balancing

Workflow

This chapter is devoted to the process of how the UGC streaming platform works, what steps the content goes through to get to the viewer's screen.

Get back to the diagram of a UGC streaming platform:

UGC platform architecture design

The black arrows in the diagram show the direction of streams, and the pink arrows indicate the direction of requests.

1. Creator <-> Publishing server <-> Central

Receive publishing

To start streaming, the creator needs to authorize and initiate stream publishing to the publishing server via SRT, WebRTC (WHIP) or RTMP, depending on the source. Here is how the creator starts streaming:

  1. The creator's request goes to the DNS server.
  2. The DNS server redirects the request to the address of one of the available publishing servers in the cluster. See below for details on DNS balancing.
  3. The publishing server sends a request to Central to authorize the creator.
  4. Once the publishing server receives a positive response, it requests the configuration for the stream from Central.
  5. Central returns the necessary configuration to the publishing server.
  6. The creator publishes the stream.

2. Publishing server -> Central -> Transcoding server

Transcoding

  1. The publishing server repackages the incoming stream into Flussonic-to-Flussonic M4S protocol and sends it to a cluster of transcoding servers.
  2. Central as a load balancer checks health states of all the servers in a cluster and redirects the stream to the least loaded server.
  3. The transcoding server receives the M4S stream, unpacks and decodes it, getting the raw video and audio.
  4. The transcoding server converts stream into a stream with multiple video tracks with different resolutions and bitrates.

Note

When receiving a stream, the Flussonic Media Server unpackages the stream and repackages it. It increases the stability of the output stream.

3. Transcoding server -> Central <-> DVR server

DVR

The DVR server accomplishes the following tasks:

  1. Requests the stream configuration from Central.
  2. Captures the M4S streams from the transcoding server, records them, and stores copies of those streams.

The DVR server also communicates with another DVR server in the cluster to replicate the archive. This way, the archive is replicated to other servers to ensure reliability and automatic data recovery after failures. See below for more information on backup.

4. DVR server <-> Central <-> Egress server <-> Viewers

Delivery to viewers

Viewers need to be authenticated and authorized to watch the stream.

  1. The viewers access the website from their devices and enter their usernames and passwords. An authentication request is then sent to the Central server.
  2. Central checks the viewers and authenticates them.
  3. After successful authentication, the viewer sends the request to the Central load balancer.
  4. The Central redirects the request to the least loaded and available egress server in the cluster.
  5. The egress server then sends a request to the Central server to authorize the viewer.
  6. Central checks if the viewer has a permission to play the stream.
  7. Based on the response received, the egress server either sends the stream to the viewer or not.

Streams are delivered in two different ways, depending on whether the stream is live or recorded:

  1. From the egress servers directly to viewers.

    When viewers request a live stream through a browser or mobile device, stream is delivered via WebRTC and MSE-LD. WebRTC and MSE-LD provide lowest delay when playing the stream.

  2. From the egress servers to CDN and then to vewers.

    When viewers request a recorded stream through a browser or mobile device, the stream is delivered via HLS and MPEG-DASH via CDN. CDN allows to deliver the content to the viewer quickly, reduce the network costs and optimize the load on servers.

Redundancy and scaling strategies

This section provides architecture design ways to build a reliable service. It's essential for a reliable service to meet these requirements:

  • Keep responding to customer requests in spite of a high demand on the service or a maintenance event.
  • Handle failures.
  • Scale in response to changes in workloads and user demands.

Cluster of publishing servers and DNS balancing

The above diagram uses a cluster of several DNS-balanced publishing servers. This approach allows to:

  • Distribute the load on several servers.
  • Ensure the creator connects to at least one active server.

To provide a redundant service when publishing SRT and RTMP streams that aren't supported by the application layer, use DNS balancing. To distribute publishing WebRTC (WHIP) streams, use the built-in Flussonic load balancer.

All the publishing servers in the cluster are the same, i.e. they are equal and interchangeable, and each of them can accept any stream. So if one of the servers in the cluster fails, another server automatically takes over.

In DNS balancing, the server to which the request will be sent to is selected according to a certain algorithm, for example, round robin. There are several ways to find out the address of the server to send the creator's request to: request it via API (recommended) or return the appropriate one using network methods. It depends on the architecture of your network.

However, DNS has a significant disadvantage—it doesn't check the server load of the servers in the cluster. That is why creators may be redirected to a server which may be currently overloaded or down.

Also keep in mind that:

  • DNS is cached for up to three days. If there are servers in the cache that are down or in a closed network, the connection timeout may be very long.
  • RTMP has no built-in redirection mechanism. Temporary solutions made to fix this don't work. However, WHIP (WebRTC HTTP Ingest Protocol) is balanced by redirects and proper made SDP.

In our case, we need a balancer that redirects requests rather than a proxy. If one publishing server fails, creator requests will be redirected to remaining active servers.

Let's consider the use of DNS balancing on a cloud platform that provides UGC service functionality for various projects.

For a creator to connect to at least one active server, at least two DNS records are needed. So for each project a separate group of servers is provided, according to server load or, for example, the geographical location of the creators.

It provides the following features:

  • Multiple DNS entries for redundancy.
  • A separate domain for each project to balance requests for publishing streams and resource allocation.

Load-balanced transcoder cluster

If a transcoding server crashes, it can cause the whole service to fail. To avoid this, follow these recommendations:

  • Use at least two transcoding servers as a cluster.
  • Provide a failover, so that if one of the servers goes down, other servers can take its workload.
  • Distribute the load evenly between the servers.

We suggest clustering with load balancing using Central.

A cluster of two or more transcoding servers can ensure uninterrupted operation of transcoders. If one of the transcoding servers fails, the other servers will be able to run processes on their side. Thus, the service will continue to run and keep working as expected. For more information about clustering, see Cluster.

To avoid overloading the transcoding servers, it is necessary to distribute requests to the cluster of transcoding servers evenly. Central load balances requests following these steps:

  1. Evaluating the load status of transcoding servers in the cluster.
  2. Directing requests to the least loaded servers.

For more information about Central and load balancing, see Flussonic load balancing.

DVR cluster with replication

A sudden outage, system failure, or any other issue can cause DVR data loss. To secure your service, you need to back up your DVR archive and copy it to other servers. We suggest using a cluster of two or more DVR servers with replication.

Replication allows to copy the recordings to other servers to accomplish the following tasks:

  • Ensure reliability.
  • Provide automatic recovery after a system failure or shutdown.

A DVR archive is stored on two or more server instances, where one is a primary server and the others are secondary servers. Replication works like so:

  1. Primary server pulls the streams from the source and stores.
  2. The secondary server pulls the streams from the primary server (see the diagram).

Flussonic provides other tools to back up a DVR archive and keep it available at all times, such as:

CDN

With the increasing number of viewers and amount of output traffic, at some point one server becomes incapable of handling the delivery of streams. As the viewers from all over the world tune in, it becomes challenging to provide them with a good and reliable service. It makes it overwhelming for an egress server to serve more and more requests at a time.

CDN (Content Delivery Network) allows you to accomplish the following objectives:

  • Increase the number of viewers without spending a lot of money on hardware.
  • Distribute the load evenly between the servers.
  • Make the content accessible for the viewers.

Having multiple servers distributed across various geographic locations, CDN provides these advantages:

  • Shortens the distance between the edge server and the viewer.
  • Minimizes latency.
  • Allows quick access to the content for the viewer.
  • Reduces the load on the egress server to keep it up and running.

To provide an efficient delivery of the content worldwide, CDN uses the following methods:

  • Caches the content in different PoPs (points of presence) to deliver it directly to the viewer.
  • Distributes the workload between multiple servers.

You can build your own CDN or use existing solutions, such as Akamai, CDNvideo, and others, to do the job. It depends on the number of viewers, their geographical location, allocated budget, and so on.

Analyze and establish requirements

When starting a project, the initial step is to perform an analysis and establish the requirements.

To establish the requirements, follow these steps:

  1. Define a niche
  2. Study your target audience
  3. Define the content and its creators
  4. Define the key features of the service, having studied the market.

Define a niche

Choose the kind of market area your service will serve to. Is it going to be health and fitness, online learning, arts and crafts, or gaming, and so on? Will it cover one use case or several ones?

Defining the niche is crucial for a UGC streaming service or platform to determine the objectives of your service or platform, identify the target audience, develop your marketing plan and further work. Research the market, see what your competitors have to offer.

Study your target audience

Define who your audience is and what they want. Answer the following questions:

  • What do your viewers have in common?
  • Why does the audience watch the content? For education purposes, entertainment, sport, an so on?
  • What devices do viewers use to watch the content? PC, mobile phone (Android, iOS), and so on?
  • What approximate number of concurrent viewers must the service or platform handle?

By answering these questions, you can identify your target audience and what they expect from your UGC streaming service or platform.

For example, the target audience of the gaming industry enjoys video games and watches walkthroughs, eSports competitions, an so on. Viewers use PCs, laptops, mobile phones (Android, iOS), tablets. It's an example of entertainment content watched by hundreds of thousands of viewers.

Define the content and its creators

Determine what content you want to provide and who will be providing this content. Determine what your target audience needs and their preferences to define what content they will watch.

Consider Twitch. Twitch was initially created as a live streaming platform for gamers. Gamers stream video game walkthroughs and gameplays, tournament organizers stream eSports competitions.

When determining what content to provide through your platform or service, rely on the target audience and its preferences.

Define the key features of the service

When creating a streaming service or a platform, provide features for your customers that meet their needs and demands.

Here are some of the common features of UGC streaming platforms:

  • for creators: stream with different tools, for example, VMix, OBS, Atemi, HDMI-RTMP converters, video editing consoles, and web browsers
  • for viewers: watch the content from any device, for example, PC, mobile phone, laptop
  • record, store and play live streams
  • multistream to social media and platforms, like YouTube, Instagram, Facebook, Twitter, Twitch
  • continuous and stable playback of content
  • upload pre-recorded streams
  • low-latency streaming
  • embedded media player
  • authorization system
  • subscription system, and so on

Depending on the goals and objectives of the platform or service, allocated budget, and available hardware and software resources, the set of features can vary.

Design the network architecture

This chapter describes how to do the following tasks:

  • Estimate the required input and output network bandwidths for a publishing server, a transcoder, a DVR, an egress server.
  • Create a network architecture diagram.

To calculate the input and output bandwidth and create a network architecture diagram, answer these questions:

Table 2. Questions to determine the network requirements

Topic Questions
Source (input) 1) What is the type of signal source (camera, software encoder, hardware encoder, browser)?
2) What are the parameters of the input stream (codec, bitrate, resolution, FPS)?
3) Which streaming protocol is used for transmitting video from the source (RTMP, SRT, WebRTC)?
4) What is the total number of incoming streams?
Transcoder 1) What are the output video stream profiles (codec, bitrate, resolution)?
2) What is the total number of output streams for distribution?
3) Is audio transcoding required? If yes, what are the output audio stream profiles (codec, bitrate)? (For example, Opus (WebRTC) -> AAC (HLS, DASH))
DVR archive (recording and storing) 1) Is stream recording required? If yes, how do you plan to use DVR (export recordings to VOD, implement catch-up, etc.)?
2) What is the required depth of the archive (one day, week, month)?
3) Where will streams be stored (cloud storage, disk, RAID)?
Egress server (output) 1) What are the types of client devices (mobile phones, PCs, and so on)?
2) What is the streaming protocol (HLS, MPEG-DASH, WebRTC, etc.) for content distribution?
3) What is the estimated number of viewers?
4) Are you planning to distribute content using your own resources or third-party CDN (Akamai or others)?
Security 1) Is authorization required? If yes, by what means: external backend or built-in Flussonic tools?
2) Is content encryption necessary?
Other requirements 1) Are audio tracks with multiple audio tracks with different languages?
2) Do you need additional control of any video parameters?
and so on.

To calculate the network bandwidth, use the following formula:

number of streams * (video bitrate + audio bitrate)

Based on the questions from Table 2, you can determine the number of publishing servers, transcoders, DVR servers, and egress servers and create a diagram.

Then move on to setting up and configuring the servers.

Example of an implementation strategy for a UGC service

This chapter shows how to develop an implementation strategy for an event broadcasting platform. Follow these steps:

  1. Analyze and establish requirements
  2. Design the network architecture diagram

Warning

We don't provide any programming code or server configuration for this project.

Analyze and establish requirements

Suppose, you want to create an event broadcasting platform.
The project goal is to reduce company's expenses on the current service.

  • Niche: event broadcasting
  • Target audience: big and small companies
  • Type of content: summits, webinars, online workshops, online corporate events, conferences

  • Key features:

    • Support VMix, OBS.
    • Record live streams and store them, so the viewers can watch them later.
    • Provide failover and backup to keep the service up and running if any issues occur.
    • Ensure that viewers with slow internet connection can watch the stream.
    • Manage the available resources efficiently at uneven server loads.
    • Make the content accessible from any mobile devices and browsers.
    • Provide statistics of a system performance.
    • Authenticate and authorize viewers and creators.

Design the network architecture

You gave the following answers to the questions from Table 2:

Topic Description
Source (input) 1) Source type: a hardware encoder or a software encoder like vMix.
2) Stream parameters: H.264, 5 000 Kbps, 1920x1080p, 25 FPS.
3) Streaming protocols: RTMP or SRT.
4) Number of input streams: up to 10.
Transcoder 1) Transcoding in three profiles: Full HD (1080p), HD (720p), SD (360p).
2) Number of output streams: up to 30.
3) Audio transcoding isn't required.
DVR archive (recording and storing) 1) Recording and storing of streams is required. Viewers will have the opportunity to watch an event wherever and whenever they choose and to download the recording to their device.
2) Up to 10 hours for all 10 streams in a separate data center. It's neccessary to be able to export the archive fragments as MP4 files.
3) S3 cloud storage stores the recorded events.
Egress server (output) 1) Client devices: PC and mobile phones.
2) Streaming protocol: HLS.
3) Estimated number of viewers: up to 100,000.
4) Using CDN Akamai to deliver the content.
Security 1) Authorization of users is required.
2) Encryption isn't neccessary.
Other requirements 1) To create streams automatically, API integration is required.

To calculate the network bandwidth, we use the formula:

number of streams * (video bitrate + audio bitrate)

  • The input and output bandwidths for the publishing server:

    10 streams * (5,000 + 192)Kbps ≈ 52 Mbps

  • The output bandwidth for the transcoder. Transcoding ten Full HD (1080p, 5,000 Kbps, H.264) streams into three video profiles each:

    • Full HD (1080p), H.264, 4,000 Kbps; AAC, 192 Kbps
    • HD (720p), H.264, 2,000 Kbps; AAC, 128 Kbps
    • SD (360p), H.264, 1,000 Kbps; AAC, 96 Kbps

    Makes:

    30 * (4,000 + 2,000 + 1,000 + 192 + 128 + 96)Kbps ≈ 223 Mbps

  • The input and output bandwidths for the DVR equal the output bandwidth for the transcoder.

  • The output bandwidth for the egress server, given that all 100,000 viewers watch the Full HD stream at the same time:

    100,000 * (4,000 + 192)Kbps ≈ 420 Gbps

Now we can make a network diagram:

Diagram 3. Network architecture diagram for an event broadcasting platform

Event streaming network architecture diagram

Four Flussonic servers are required:

  • Transcoding server №1 and transcoding server №2: receive and transcode the streams.
  • Egress server №1 and egress server №2: record the streams and deliver the content to the viewers.

Authorization

The platform authorizes viewers by their specific token:

  1. A website generates a unique token for each viewer so that the viewer cannot pass it on to others.
  2. Flussonic checks the token, the IP address and some other parameters.

This methos is great for private events.

Transcoding servers

Transcoding server №1 and №2 receive RTMP and SRT streams and transcode them.

Transcoding server №1 is the main server, and transcoding server №2 is the standby one. This way we can cover these scenarios:

  • If the main server becomes unavailable, the standby server takes over.
  • If both main and standby servers become unavailable, the fallback video file will start on a loop until one of the servers goes back online. The fallback video will also be written to the archive.

Egress servers

Egress server №1 and egress server №2 accomplish these tasks:

  • Record live events and write them to the archive.
  • Deliver the streams to viewers.

To record and store the live streams, the system works as follows:

  1. Egress server №1 and egress server №2 capture M4S streams from the transcoding server and record them.
  2. After the live event is finished, Flussonic automatically uploads the recordings as MP4 files to S3 cloud storage.

To speed up the delivery of VOD, we enable SSD caching. Here is how it works:

  1. The first time an MP4 file with a live stream recording is requested, the file will be cached locally on the SSD.
  2. All subsequent requests will be served from this SSD.
  3. If the cached file isn't accessed for 24 hours, then Flussonic removes the file from cache.
  4. If the total size of files stored in cache exceeds 100 GB, then Flussonic deletes files from the cache, starting with the oldest one.

To distribute the load between the servers, we use a hybrid approach:

  • Load balancer on the side of CDN Akamai handles play requests for large public events. CDN Akamai captures the HLS and LL-HLS streams from the egress server №1 and egress server №2.
  • Flussonic load balancer distributes the load between the two egress servers for small events. In this case, egress server №1 serves as a load balancer.

To deliver the streams to viewers we use HLS and LL-HLS streaming protocols:

  • LL-HLS for live streams
  • HLS for VOD

Glossary

User-generated Content (UGC)

any form of content, for example, videos and images, created by people and published online.

UGC platform

A software-as-a-service (SaaS) solution used to collect and manage the content. It connects a creator and a viewer through content. A crucial feature of a UGC platform is that it uses content creators to make the content, when IPTV/OTT platforms are content providers themselves.

Publishing server

A server that receives a stream from the creator.

Transcoding server

A server that transcodes a stream into an array of streams at different resolutions and bitrates.

DVR server

A server that takes incoming video streams, records and stores them, and then sends them on demand.

Egress server

A server that delivers the content to the viewers.

Cluster

A group of servers working together to perform common tasks.

DNS balancing

A method of assigning several IP addresses to the same domain name, which allows you to direct creator requests to different servers when accessing the same domain.