Flussonic Media Server Documentation

Dynamic Chat Rooms

Flussonic Media Server has a mechanism for creating a dynamic mosaic that shows multiple streams as one with video and audio mixing. An example of using such a mosaic is creating your own video or audio chat from streams coming to the Flussonic server from client devices.

The mosaic is called dynamic because participants (published video streams) can be added and removed programmatically while the chat is running.

In version 20.08, a chat consisting only of audio tracks is possible. Video will be supported in future versions.

The mosaic is designed for streams-publications from client cameras with the following characteristics: video — H.264, audio — Opus. The mosaic is based on the existing in Flussonic publishing and playing via WebRTC. It is played in our WebRTC player.

The mosaic stream receives audio (and video) frames from the participating streams. Then it mixes audio is mixed — the track all is for the audience, all the participants it can be heard; the track STREAM_NAME (name of a participant) — for this participant, you can hear all the participants, except STREAM_NAME.

Video frames are displayed as a grid in a single shared graphic buffer of a certain size, and this buffer is H.264 encoded and delivered as a single frame.

In future versions, it is planned to add settings to adjust the size of the graphic buffer and the parameters of the grid that shows a mosaic.

Configuration and usage

One mosaic — one chat room.

1) In the Flussonic configuration file, add the stream that will contain the mosaic of published streams:

stream ROOM_NAME  {
  url mosaic2:// bitrate=128;
}

For the mosaic the following options are available:

  • (optional) disable_video=true/false (true by default) - show the video from cameras of chat participants.

  • (optional) bitrate, samplerate, samples - paramenters of the Opus codec for audio: (the default values are 64k, 48000, 960).

Published streams from client devices that publish to the chat can be added dynamically using the API, manually adding them to the configuration is not necessary.

2) Then use the API to set the mosaic settings:

http://user:pass@flussonic.url/flussonic/api/dynamic_mosaic/
  • Add a chat:

    `curl -X POST -d '{"command" : "add_room", "args" : ["ROOM_NAME"]}' 'http://user:pass@FLUSSONIC-IP/flussonic/api/dynamic_mosaic/'

  • Add a chat, working with audio with the follow parameters: bitrate=64 samplerate=4800 samples=480:

    `curl -X POST -d '{"command" : "add_room", "args" : ["ROOM_NAME","64","48000","480"]}' 'http://user:pass@FLUSSONIC-IP/flussonic/api/dynamic_mosaic/'

  • Statistics:

    `curl -X POST -d '{"command" : "stats", "args" : []}' 'http://user:pass@FLUSSONIC-IP/flussonic/api/dynamic_mosaic/'

  • Delete a chat:

    `curl -X POST -d '{"command" : "remove_room", "args" : ["ROOM_NAME"]}' 'http://user:pass@FLUSSONIC-IP/flussonic/api/dynamic_mosaic/'

  • List rooms:

    `curl -X POST -d '{"command" : "list_room", "args" : []}' 'http://user:pass@FLUSSONIC-IP/flussonic/api/dynamic_mosaic/'

An example of using the API is shown in the demo application that comes with Flussonic (see below).

3) Using a separate room API, a client device can join this chat. In this case, a regular publishing stream (url publish: //) from the client's camera will be created in the Flussonic configuration, and this stream will be added to the mosaic.

  • Add a participant

    `curl -X POST -d '{"command" : "add_stream", "args" : ["STREAM_NAME"]}' 'http://user:pass@FLUSSONIC-IP/flussonic/api/dynamic_mosaic/room'

    Returns JSON code similar to {"track_id" : "a3"} — this is the number of the audio track which must be specified on the client player's side in order to hear the chat room.

  • Participant leaves

    `curl -X POST -d '{"command" : "remove_stream", "args" : ["STREAM_NAME"]}' 'http://user:pass@FLUSSONIC-IP/flussonic/api/dynamic_mosaic/room'

    Returns the stream's audio track number.

  • Statistics

    `curl -X POST -d '{"command" : "stat", "args" : []}' 'http://user:pass@FLUSSONIC-IP/flussonic/api/dynamic_mosaic/room'

    Returns JSON code similar to {"count" : 4, "clients" : ["all", "STREAM_NAME", ...]}

  • The mute video function is not used yet because Flussonic 20.08 supports the audio-only mosaic.

    `curl -X POST -d '{"command" : "mute_video", "args" : ["STREAM_NAME"]}' 'http://user:pass@FLUSSONIC-IP/flussonic/api/dynamic_mosaic/room'

    Returns the stream's audio track number.

  • The mute audio function is implemented both on the server side and on the client (player) side. In the demo app, the client-side mute-audio is demonstrated.

    `curl -X POST -d '{"command" : "mute-audio", "args" : ["STREAM_NAME"]}' 'http://user:pass@FLUSSONIC-IP/flussonic/api/dynamic_mosaic/room'

    Returns the stream's audio track number.

The demo application

In the demo app, all the API commands listed above are used (except remove_room and mute_video).

To run the demo app, go to the Flussonic's command line

/etc/init.d/flussonic shell

and run

chat:start().

The web interface for mosaics management will appear at https://FLUSSONIC-IP:9000.