Flussonic Media Server documentation

Fine-Tuning Flussonic Media Server and the Operating System

This section describes certain common issues and techniques of tweaking the operating system and Flussonic Media Server software for working under high load.

UDP capture setup Anchor Anchor x2

For ingesting UDP multicast streams the amount of memory allocated to UDP buffers should be increased:

sysctl -w net.core.rmem_max=1048576
sysctl -w net.core.rmem_default=1048576
sysctl -w net.ipv4.udp_mem="8388608 12582912 16777216"

Note that these settings will stay only until system reboot. In order to make those setting persistent open the file /etc/sysctl.conf in an editor and add the following lines in the end:

net.core.rmem_max = 1048576
net.core.rmem_default=1048576
net.ipv4.udp_mem = 8388608 12582912 16777216

Then execute the command sudo sysctl -p to apply the changes.

Working with a large amount of memory Anchor Anchor x2

When more than 60GB of memory is available, we recommend allocating 10GB to the system:

sysctl vm.min_free_kbytes=10240000

TCP/IP stack setup Anchor Anchor x2

If you intend to use Flussonic Media Server for broadcasting at more than 3-4 Gbit/s, you might want to fine-tune the system's TCP/IP stack.

First, you will need to allocate more memory to connection buffers:

sysctl -w net.core.wmem_max=16777216
sysctl -w net.ipv4.tcp_wmem="4096 4194394 16777216"
sysctl -w net.ipv4.tcp_congestion_control=htcp
sysctl -w net.ipv4.tcp_slow_start_after_idle=0

Note that these settings will stay only until system reboot. In order to make those setting persistent open the file /etc/sysctl.conf in an editor and add the following lines in the end:

net.core.wmem_max = 16777216
net.ipv4.tcp_wmem = 4096 4194394 16777216

Then execute the command sudo sysctl -p to apply the changes.

You will also need to change the network adapter's settings: ifconfig eth0 txqueuelen 10000.

Make sure to check the adapter's driver version. Using the latest version is recommended. Use ethtool to find the version of the driver and the firmware:

# ethtool -i eth2

driver: ixgbe
version: 3.15.1
firmware-version: 0x61c10001
bus-info: 0000:04:00.0

Important! If the firmware file in the /lib/firmware directory is updated, the server must be rebooted. The old firmware version may remain. Do not forget to run the update-initramfs utility before restarting the server.

Configuring network adapter Anchor Anchor x2

Configuring interrupts

Modern 10 Gigabit network adapters support multiple queues for incoming and outgoing packets. Sometimes these queues must be manually linked to different CPU cores.

Without this optimization trick the entire networking subsystem of the server will use only one CPU core. This is how it looks like:

# cat /proc/interrupts
           CPU0   CPU1   CPU2   CPU3   CPU4   CPU5   CPU6   CPU7
  0:       2097      0      0      0      0      0      0      0  IR-IO-APIC      timer
...
 66: 2072120005      0      0      0      0      0      0      0  IR-PCI-MSI      eth2-TxRx-0
 67:    1562779      0      0      0      0      0      0      0  IR-PCI-MSI      eth2-TxRx-1
 68:    1830725      0      0      0      0      0      0      0  IR-PCI-MSI      eth2-TxRx-2
 69:    1504396      0      0      0      0      0      0      0  IR-PCI-MSI      eth2-TxRx-3
 70:    5112538      0      0      0      0      0      0      0  IR-PCI-MSI      eth2-TxRx-4
 71:    2229416      0      0      0      0      0      0      0  IR-PCI-MSI      eth2-TxRx-5
 72:    1686551      0      0      0      0      0      0      0  IR-PCI-MSI      eth2-TxRx-6
 73:    1217916      0      0      0      0      0      0      0  IR-PCI-MSI      eth2-TxRx-7
 74:       2358      0      0      0      0      0      0      0  IR-PCI-MSI      eth2

For Intel adapters, the manufacturer provides the set_irq_affinity script, which distributes the queues to different cores. After running the script, the interrupts data looks like this:

# cat /proc/interrupts
           CPU0       CPU1       CPU2       CPU3       CPU4       CPU5       CPU6       CPU7
  0:       2097          0          0          0          0          0          0          0  IR-IO-APIC  timer
...
 66: 2072120005          0          0          0          0          0          0          0  IR-PCI-MSI  eth2-TxRx-0
 67:    1562779 1162738082          0          0          0          0          0          0  IR-PCI-MSI  eth2-TxRx-1
 68:    1830725          0 1133908105          0          0          0          0          0  IR-PCI-MSI  eth2-TxRx-2
 69:    1504396          0     177620 1123678951          0          0          0          0  IR-PCI-MSI  eth2-TxRx-3
 70:    5112538          0          0          0 1638450740          0          0          0  IR-PCI-MSI  eth2-TxRx-4
 71:    2229416     130189          0          0          0 1441511712          0          0  IR-PCI-MSI  eth2-TxRx-5
 72:    1686551          0          0          0          0          0 1402472725          0  IR-PCI-MSI  eth2-TxRx-6
 73:    1217916          0          0      66145          0          0          0 1380402032  IR-PCI-MSI  eth2-TxRx-7
 74:       2358          0          0          0          0          0          0          0  IR-PCI-MSI  eth2

This setting becomes critical when the traffic reaches the the vicinity of 3-5 Gbit/s.

Configuring the connection to a switch

If you connect server network adapter to a switch, please check that both sides have compatible settings. You should either use "auto select" settings on both sides, or strictly the same speed and duplex.

Optimization of the server for Video On Demand is discussed in detail in a dedicated section.