Skip to content

Fine-tuning Flussonic Media Server and the operating system

This section describes certain common issues and techniques of tweaking the operating system and Flussonic Media Server software for working under high load.

UDP capture setup

Flussonic automatically tunes some network setting to optimize the ingest of UDP streams including WebRTC and multicast. If this is not acceptable for you, set the DO_NOT_DO_NET_TUNING environment variable and make the settings manually:

To pass the DO_NOT_DO_NET_TUNING environment variable to Flussonic, execute:

systemctl edit flussonic

Add the following line to the opened file:

[Service]
Environment="DO_NOT_DO_NET_TUNING=true"

Save the changes.

Now you'll need to increase the amount of memory allocated to UDP buffers:

sysctl -w net.core.rmem_max=1048576
sysctl -w net.core.rmem_default=1048576
sysctl -w net.ipv4.udp_mem="8388608 12582912 16777216"

Note that these settings will stay only until system reboot. In order to make those setting persistent open the file /etc/sysctl.conf in an editor and add the following lines in the end:

net.core.rmem_max = 1048576
net.core.rmem_default=1048576
net.ipv4.udp_mem = 8388608 12582912 16777216

To apply the changes, execute the command:

sudo sysctl -p

Working with a large amount of memory

When more than 60GB of memory is available, we recommend allocating 10GB to the system:

sysctl vm.min_free_kbytes=10240000

TCP/IP stack setup

If you intend to use Flussonic Media Server for broadcasting at more than 3-4 Gbit/s, you might want to fine-tune the system's TCP/IP stack.

First, you will need to allocate more memory to connection buffers:

sysctl -w net.core.wmem_max=16777216
sysctl -w net.ipv4.tcp_wmem="4096 4194394 16777216"
sysctl -w net.ipv4.tcp_congestion_control=htcp
sysctl -w net.ipv4.tcp_slow_start_after_idle=0

Note that these settings will stay only until system reboot. In order to make those setting persistent open the file /etc/sysctl.conf in an editor and add the following lines in the end:

net.core.wmem_max = 16777216
net.ipv4.tcp_wmem = 4096 4194394 16777216

Then execute the command sudo sysctl -p to apply the changes.

You will also need to change the network adapter's settings: ifconfig eth0 txqueuelen 10000.

Make sure to check the adapter's driver version. Using the latest version is recommended. Use ethtool to find the version of the driver and the firmware:

ethtool -i eth2

The output will be like:

driver: ixgbe
version: 3.15.1
firmware-version: 0x61c10001
bus-info: 0000:04:00.0

Warning

If the firmware file in the /lib/firmware directory is updated, the server must be rebooted. The old firmware version may remain. Do not forget to run the update-initramfs utility before restarting the server.

Configuring network adapter

Configuring interrupts

Modern 10 Gigabit network adapters support multiple queues for incoming and outgoing packets. Sometimes these queues must be manually linked to different CPU cores.

Without this optimization trick the entire networking subsystem of the server will use only one CPU core. This is how it looks like:

cat /proc/interrupts

The output will look as follows:

           CPU0   CPU1   CPU2   CPU3   CPU4   CPU5   CPU6   CPU7
  0:       2097      0      0      0      0      0      0      0  IR-IO-APIC      timer
...
 66: 2072120005      0      0      0      0      0      0      0  IR-PCI-MSI      eth2-TxRx-0
 67:    1562779      0      0      0      0      0      0      0  IR-PCI-MSI      eth2-TxRx-1
 68:    1830725      0      0      0      0      0      0      0  IR-PCI-MSI      eth2-TxRx-2
 69:    1504396      0      0      0      0      0      0      0  IR-PCI-MSI      eth2-TxRx-3
 70:    5112538      0      0      0      0      0      0      0  IR-PCI-MSI      eth2-TxRx-4
 71:    2229416      0      0      0      0      0      0      0  IR-PCI-MSI      eth2-TxRx-5
 72:    1686551      0      0      0      0      0      0      0  IR-PCI-MSI      eth2-TxRx-6
 73:    1217916      0      0      0      0      0      0      0  IR-PCI-MSI      eth2-TxRx-7
 74:       2358      0      0      0      0      0      0      0  IR-PCI-MSI      eth2

For Intel adapters, the manufacturer provides the set_irq_affinity script, which distributes the queues to different cores. After running the script, the interrupts data looks like this:

           CPU0       CPU1       CPU2       CPU3       CPU4       CPU5       CPU6       CPU7
  0:       2097          0          0          0          0          0          0          0  IR-IO-APIC  timer
...
 66: 2072120005          0          0          0          0          0          0          0  IR-PCI-MSI  eth2-TxRx-0
 67:    1562779 1162738082          0          0          0          0          0          0  IR-PCI-MSI  eth2-TxRx-1
 68:    1830725          0 1133908105          0          0          0          0          0  IR-PCI-MSI  eth2-TxRx-2
 69:    1504396          0     177620 1123678951          0          0          0          0  IR-PCI-MSI  eth2-TxRx-3
 70:    5112538          0          0          0 1638450740          0          0          0  IR-PCI-MSI  eth2-TxRx-4
 71:    2229416     130189          0          0          0 1441511712          0          0  IR-PCI-MSI  eth2-TxRx-5
 72:    1686551          0          0          0          0          0 1402472725          0  IR-PCI-MSI  eth2-TxRx-6
 73:    1217916          0          0      66145          0          0          0 1380402032  IR-PCI-MSI  eth2-TxRx-7
 74:       2358          0          0          0          0          0          0          0  IR-PCI-MSI  eth2

This setting becomes critical when the traffic reaches the vicinity of 3-5 Gbit/s.

Configuring the connection to a switch

If you connect server network adapter to a switch, please check that both sides have compatible settings. You should either use auto select settings on both sides, or strictly the same speed and duplex.

Optimizing the server for VOD

Optimization of the server for Video On Demand is discussed in detail in a dedicated section.

Switching the CPU to the high performance mode

In Linux the scaling_governor knob is in power save mode by default. In this case, the server does not use all of its hardware resources. For the server to work in high performance mode, do the following:

Disable the ondemand controller:

systemctl disable ondemand

Reboot the server:

reboot

Check the current value of scaling_governor:

cat /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor