Flussonic Media Server documentation

Fine-tuning Flussonic and OS

This section discusses certain common issues and techniques of tweaking the operating system and Flussonic for greater workloads.

UDP capture setup Anchor Anchor x2

For capturing data via UDP under Linux, the amount of memory allocated to UDP buffers should be increased:

sysctl -w net.core.rmem_max=1048576
sysctl -w net.core.rmem_default=1048576
sysctl -w net.ipv4.udp_mem="8388608 12582912 16777216"
sysctl -w net.ipv4.tcp_congestion_control=htcp
sysctl -w net.ipv4.tcp_slow_start_after_idle=0

Note that these settings will persist only until reload. In order to save them permanently, edit the file /etc/sysctl.conf by adding at the end of it the following:

net.core.rmem_max = 1048576
net.core.rmem_default=1048576
net.ipv4.udp_mem = 8388608 12582912 16777216
and then running the sudo sysctl -p command to apply the changes.

Working with a large amount of memory Anchor Anchor x2

When more than 60GB of memory is available, we recommend allocating 10GB to the system:

sysctl vm.min_free_kbytes=10240000

TCP/IP stack setup Anchor Anchor x2

If you intend to use Flussonic for broadcasting at more than 3-4 Gbit/s, you might want to fine-tune the system's TCP/IP stack.

First, you will need to allocate more memory to connection buffers:

sysctl -w net.core.wmem_max=16777216
sysctl -w net.ipv4.tcp_wmem="4096 4194394 16777216"

Note that these settings will persist only until reload. In order to save them permanently, edit the file /etc/sysctl.conf by adding at the end of it the following:

net.core.wmem_max = 16777216
net.ipv4.tcp_wmem = 4096 4194394 16777216

and then running the sudo sysctl -p command to apply the changes.

You will also need to change the network adapter's settings: ifconfig eth0 txqueuelen 10000.

Make sure to check the adapter's driver version. Using the latest version is recommended. You can find out the version of the driver and the firmware by typing the following:

# ethtool -i eth2

driver: ixgbe
version: 3.15.1
firmware-version: 0x61c10001
bus-info: 0000:04:00.0

Important. If the firmware file in the `/lib/firmware` directory is updated, the server must be reloaded. The old firmware version may remain. Do not forget to run the `update-initramfs` utility before restarting the server.

Configuring network adapter Anchor Anchor x2

Configuring interrupts

Contemporary 10 gigabit network adapters support multiple queues for incoming and outgoing packets. Sometimes these queues must be manually associated with different CPU cores.

If this optimization trick has not been performed, the server processes the entire networking subsystem using only one core. This is how it looks like:

# cat /proc/interrupts
           CPU0   CPU1   CPU2   CPU3   CPU4   CPU5   CPU6   CPU7
  0:       2097      0      0      0      0      0      0      0  IR-IO-APIC      timer
...
 66: 2072120005      0      0      0      0      0      0      0  IR-PCI-MSI      eth2-TxRx-0
 67:    1562779      0      0      0      0      0      0      0  IR-PCI-MSI      eth2-TxRx-1
 68:    1830725      0      0      0      0      0      0      0  IR-PCI-MSI      eth2-TxRx-2
 69:    1504396      0      0      0      0      0      0      0  IR-PCI-MSI      eth2-TxRx-3
 70:    5112538      0      0      0      0      0      0      0  IR-PCI-MSI      eth2-TxRx-4
 71:    2229416      0      0      0      0      0      0      0  IR-PCI-MSI      eth2-TxRx-5
 72:    1686551      0      0      0      0      0      0      0  IR-PCI-MSI      eth2-TxRx-6
 73:    1217916      0      0      0      0      0      0      0  IR-PCI-MSI      eth2-TxRx-7
 74:       2358      0      0      0      0      0      0      0  IR-PCI-MSI      eth2

For Intel adapters, the manufacturer provides the set_irq_affinity script, which distributes the queues to different cores. After running the script, the interrupts data looks like this:

# cat /proc/interrupts
           CPU0       CPU1       CPU2       CPU3       CPU4       CPU5       CPU6       CPU7
  0:       2097          0          0          0          0          0          0          0  IR-IO-APIC  timer
...
 66: 2072120005          0          0          0          0          0          0          0  IR-PCI-MSI  eth2-TxRx-0
 67:    1562779 1162738082          0          0          0          0          0          0  IR-PCI-MSI  eth2-TxRx-1
 68:    1830725          0 1133908105          0          0          0          0          0  IR-PCI-MSI  eth2-TxRx-2
 69:    1504396          0     177620 1123678951          0          0          0          0  IR-PCI-MSI  eth2-TxRx-3
 70:    5112538          0          0          0 1638450740          0          0          0  IR-PCI-MSI  eth2-TxRx-4
 71:    2229416     130189          0          0          0 1441511712          0          0  IR-PCI-MSI  eth2-TxRx-5
 72:    1686551          0          0          0          0          0 1402472725          0  IR-PCI-MSI  eth2-TxRx-6
 73:    1217916          0          0      66145          0          0          0 1380402032  IR-PCI-MSI  eth2-TxRx-7
 74:       2358          0          0          0          0          0          0          0  IR-PCI-MSI  eth2

This setting becomes critical when the traffic reaches the the vicinity of 3-5 Gbit/s.

Configuring connection to switch

If you connect server network adapter to a switch, please check that both sides have compatible settings. You should either use "auto select" settings on both sides, or strictly the same speed and duplex.

Optimization of the server for Video On Demand is discussed in detail in a dedicated section.