Fine-tuning Flussonic Media Server and the operating system¶
This section describes certain common issues and techniques of tweaking the operating system and Flussonic Media Server software for working under high load.
UDP capture setup¶
Flussonic automatically tunes some network setting to optimize the ingest of UDP streams including WebRTC and multicast. If this is not acceptable for you, set the DO_NOT_DO_NET_TUNING environment variable and make the settings manually:
To pass the DO_NOT_DO_NET_TUNING environment variable to Flussonic, execute:
systemctl edit flussonic
Add the following line to the opened file:
[Service]
Environment="DO_NOT_DO_NET_TUNING=true"
Save the changes.
Now you'll need to increase the amount of memory allocated to UDP buffers:
sysctl -w net.core.rmem_max=1048576
sysctl -w net.core.rmem_default=1048576
sysctl -w net.ipv4.udp_mem="8388608 12582912 16777216"
Note that these settings will stay only until system reboot.
In order to make those setting persistent open the file /etc/sysctl.conf in an editor and add the following lines in the end:
net.core.rmem_max = 1048576
net.core.rmem_default=1048576
net.ipv4.udp_mem = 8388608 12582912 16777216
To apply the changes, execute the command:
sudo sysctl -p
Working with a large amount of memory¶
When more than 60GB of memory is available, we recommend allocating 10GB to the system:
sysctl vm.min_free_kbytes=10240000
TCP/IP stack setup¶
If you intend to use Flussonic Media Server for broadcasting at more than 3-4 Gbit/s, you might want to fine-tune the system's TCP/IP stack.
First, you will need to allocate more memory to connection buffers:
sysctl -w net.core.wmem_max=16777216
sysctl -w net.ipv4.tcp_wmem="4096 4194394 16777216"
sysctl -w net.ipv4.tcp_congestion_control=htcp
sysctl -w net.ipv4.tcp_slow_start_after_idle=0
Note that these settings will stay only until system reboot.
In order to make those setting persistent open the file /etc/sysctl.conf in an editor and add the following lines in the end:
net.core.wmem_max = 16777216
net.ipv4.tcp_wmem = 4096 4194394 16777216
Then execute the command sudo sysctl -p to apply the changes.
You will also need to change the network adapter's settings: ifconfig eth0 txqueuelen 10000.
Make sure to check the adapter's driver version. Using the latest version is recommended. Use ethtool to find the version of the driver and the firmware:
ethtool -i eth2
The output will be like:
driver: ixgbe
version: 3.15.1
firmware-version: 0x61c10001
bus-info: 0000:04:00.0
Warning
If the firmware file in the /lib/firmware directory is updated, the server must be rebooted. The old firmware version may remain. Do not forget to run the update-initramfs utility before restarting the server.
Configuring network adapter¶
Configuring interrupts¶
Modern 10 Gigabit network adapters support multiple queues for incoming and outgoing packets. Sometimes these queues must be manually linked to different CPU cores.
Without this optimization trick the entire networking subsystem of the server will use only one CPU core. This is how it looks like:
cat /proc/interrupts
The output will look as follows:
CPU0 CPU1 CPU2 CPU3 CPU4 CPU5 CPU6 CPU7
0: 2097 0 0 0 0 0 0 0 IR-IO-APIC timer
...
66: 2072120005 0 0 0 0 0 0 0 IR-PCI-MSI eth2-TxRx-0
67: 1562779 0 0 0 0 0 0 0 IR-PCI-MSI eth2-TxRx-1
68: 1830725 0 0 0 0 0 0 0 IR-PCI-MSI eth2-TxRx-2
69: 1504396 0 0 0 0 0 0 0 IR-PCI-MSI eth2-TxRx-3
70: 5112538 0 0 0 0 0 0 0 IR-PCI-MSI eth2-TxRx-4
71: 2229416 0 0 0 0 0 0 0 IR-PCI-MSI eth2-TxRx-5
72: 1686551 0 0 0 0 0 0 0 IR-PCI-MSI eth2-TxRx-6
73: 1217916 0 0 0 0 0 0 0 IR-PCI-MSI eth2-TxRx-7
74: 2358 0 0 0 0 0 0 0 IR-PCI-MSI eth2
For Intel adapters, the manufacturer provides the set_irq_affinity script, which distributes the queues to different cores. After running the script, the interrupts data looks like this:
CPU0 CPU1 CPU2 CPU3 CPU4 CPU5 CPU6 CPU7
0: 2097 0 0 0 0 0 0 0 IR-IO-APIC timer
...
66: 2072120005 0 0 0 0 0 0 0 IR-PCI-MSI eth2-TxRx-0
67: 1562779 1162738082 0 0 0 0 0 0 IR-PCI-MSI eth2-TxRx-1
68: 1830725 0 1133908105 0 0 0 0 0 IR-PCI-MSI eth2-TxRx-2
69: 1504396 0 177620 1123678951 0 0 0 0 IR-PCI-MSI eth2-TxRx-3
70: 5112538 0 0 0 1638450740 0 0 0 IR-PCI-MSI eth2-TxRx-4
71: 2229416 130189 0 0 0 1441511712 0 0 IR-PCI-MSI eth2-TxRx-5
72: 1686551 0 0 0 0 0 1402472725 0 IR-PCI-MSI eth2-TxRx-6
73: 1217916 0 0 66145 0 0 0 1380402032 IR-PCI-MSI eth2-TxRx-7
74: 2358 0 0 0 0 0 0 0 IR-PCI-MSI eth2
This setting becomes critical when the traffic reaches the vicinity of 3-5 Gbit/s.
Configuring the connection to a switch¶
If you connect server network adapter to a switch, please check that both sides have compatible settings. You should either use auto select settings on both sides, or strictly the same speed and duplex.
Optimizing the server for VOD¶
Optimization of the server for Video On Demand is discussed in detail in a dedicated section.
Switching the CPU to the high performance mode¶
In Linux the scaling_governor knob is in power save mode by default. In this case, the server does not use all of its hardware resources. For the server to work in high performance mode, do the following:
Disable the ondemand controller:
systemctl disable ondemand
Reboot the server:
reboot
Check the current value of scaling_governor:
cat /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor