Flussonic Media Server documentation

Cluster management via SQL API

Flussonic allows you to manage streams (create, update, delete and select) across whole cluster of servers via single entry point.

In this article we will explain how to do it with all details.

Important!

You do not need to install MySQL server! Flussonic does not work with MySQL server.

Flussonic behaves as if it is MySQL server. When you specify mysql 3306; in your config file, Flussonic will bind to this port and work as a MySQL server (with specific functionality).

Do not install MySQL server!!

Usually mysql client tries to connect to mysql server via Unix domain socket. When you tell him to connect to localhost it tries to open something like /tmp/mysql.sock or whatever else. Flussonic listens strictly on TCP socket, so when you try to do it on localhost, you need to use 127.0.0.1. Usually it is alias, but in case of mysql client, there is a difference.

Servers Anchor Anchor x2

First you need to read about concept of servers in cluster. Each Flussonic knows about himself and also you can tell that there are other servers in cluster by directive server

Servers in cluster use cluster_key to authorize on each other. You need to declare cluster_key in config file to configure cluster key on this Flussonic. When you add cluster_key to server directive, you tell Flussonic which cluster_key it must use to connect to other server. By default Flussonic will use its cluster key to connect with peers.

If you have two servers: srv01.cdn.local and srv02.cdn.local then can configure them in following way:

srv01.cdn.local:

http 80;
mysql 3306;
edit_auth admin secretpass;
cluster_key mcXpNJyZX3mSE3;
server srv01.cdn.local;
server srv02.cdn.local {
  cluster_key wjFxSiSG6EC6e2;
}

srv02.cdn.local:

http 80;
cluster_key wjFxSiSG6EC6e2;

When you configure it in such way, srv01 will periodically ping srv02 and fetch status of streams on it.

Now let's take a look at how can you achieve the same confguration with SQL API:

$ mysql -u admin -psecretpass -h srv01.cdn.local cluster
mysql> INSERT INTO servers (hostname,cluster_key) VALUES ('srv02.cdn.local','wjFxSiSG6EC6e2');
Query OK, 1 row affected (0.36 sec)

mysql> SELECT * FROM servers;
+----------------------+-----------+--------------------------------------+------+----------------+--------+------+
| hostname             | available | id                                   | port | cluster_key    | uptime | load |
+----------------------+-----------+--------------------------------------+------+----------------+--------+------+
| srv02.cdn.local      |         1 | bcac6edd-46a7-40cc-970a-5da3bec8d94b |   80 | wjFxSiSG6EC6e2 |  78715 | NULL |
| localhost            |         1 | cd30c5a4-3dda-4ac1-a2eb-a9b765918082 |   80 | mcXpNJyZX3mSE3 |   NULL |    0 |
+----------------------+-----------+--------------------------------------+------+----------------+--------+------+

Now you get the same configuration as described above. If you see id than it means that current Flussonic could connect to remote Flussonic and establish peering. You will see uptime and even load (output bandwidth utilization in percents).

You can delete this server from your config:

mysql> DELETE FROM servers WHERE hostname='srv02.cdn.local';
Query OK, 1 row affected (0.68 sec)

Now this server is deleted and you will not see streams from this server.

Stream management Anchor Anchor x2

Whole idea of this management interface is that you can get access to all streams on all of your servers in single place via regular SQL API.

Also we give you ability to store some metadata directly in stream confguration section, so perhaps you don't need to have a "streams" table in your management system.

Stream is identified by name column and have column server with hostname of server on which stream is located.

If you create stream without server specification, it will be created on the server to which you are connected. If you specify server among fields, it will be created on the specified server.

You don't need to specify server when you update, delete or select stream because Flussonic will automatically find on which server stream is running and do required actions on that server.

Server structure Anchor Anchor x2

Server table contains the following columns:

Field Type
hostname string
available bool
id string
port integer
https_port integer
rtmp_port integer
rtsp_port integer
cluster_key string
uptime integer
meta string
bandwidth_usage integer outbound traffic, % from total_bandwidth
cpu_usage integer CPU usage in %
memory_usage integer memory usage, % from RAM
total_clients integer
online_streams integer
total_streams integer
version string
build integer
is_remote bool
is_peer bool
is_source bool
error string
fetched_at integer
vsaas bool internal
vsaas_endpoint string internal
rproxy bool internal
rproxy_streampoint_key string internal
rproxy_endpoint_auth string internal

Stream structure Anchor Anchor x2

Stream table is rather wide because each stream has a lot of information.

Field Type
name string
title string
provider string
static bool
pulse_off bool
position integer
disabled bool
retry_limit integer
source_timeout integer
password string
publish_enabled bool
prefix string
max_bitrate integer
on_publish string
motion_detector_enabled bool
motion_detector_notify string
motion_detector_tags string
transcoder string
dvr_root string
hls_off bool
hds_off bool
rtmp_off bool
rtsp_off bool
dash_off bool
mpegts_off bool
webrtc_off bool
m4s_off bool
m4f_off bool
mseld_off bool
url_prefix string
segment_count integer
gop_duration integer
prepush integer
backup string
add_audio_only bool
max_sessions integer
thumbnails_enabled bool
thumbnails_url string
clients_timeout integer
push string
external_cache string internal
auth_url string
domains string
allowed_countries string
disallowed_countries string
glue_ts bool
vsaas bool internal
iptv bool internal
comment string
coordinates string internal
postal_address string internal
owner string internal
auth_token string internal
mobile_token string internal
access string internal
onvif_url string internal
onvif_profile string internal
can_ptz bool internal
dvr_protected bool internal
agent_id string internal
agent_pin string internal
agent_key string internal
agent_model string internal
agent_serial string internal
registered_at integer internal
program_id integer internal
stream_id integer internal
extra string internal
dvr string
urls string
cluster_ingest bool
alive bool
remote bool
source_hostname string
dvr_enabled bool
lifetime integer internal
start_running_at integer
ts_delay integer internal
retry_count integer
client_count integer
last_dts integer internal
last_dts_at integer internal
last_access_at integer
input_error_rate integer internal
bytes_in integer
bytes_out integer
out_bandwidth integer
bufferings integer internal
bitrate integer
source_error string source error
url string
current_agent_id string internal
agent_status string internal
server string
published_from string publication source IP address
published_via string publication format
dvr_only bool internal
running_transcoder bool
dvr_replication integer
dvr_replication_running bool

You can see that not all configuration options are here yet, we are working on it.

Creation Anchor Anchor x2

when you first launch fresh cluster, you will have no streams:

mysql> select * from streams;
Empty set (0.00 sec)

Let's create now something. Here and later we assume that you have configured on all servers:

file vod {
  path priv;
}

and file bunny.mp4 in this priv path.

mysql> insert into streams (name,urls,dvr) values  ('bunny1','file://vod/bunny.mp4','movies 2d 20G');
Query OK, 1 row affected (0.02 sec)
mysql> select name,static,urls,lifetime,server from streams;
+--------+--------+----------------------+----------+-----------+
| name   | static | urls                 | lifetime | server    |
+--------+--------+----------------------+----------+-----------+
| bunny1 |      1 | file://vod/bunny.mp4 |    27029 | localhost |
+--------+--------+----------------------+----------+-----------+
1 row in set (0.01 sec)

Now let's make some cluster work:

mysql> insert into streams (name,urls,dvr,server) values  ('bunny2','file://vod/bunny.mp4','movies 2d 20G','srv02.cdn.local');
Query OK, 1 row affected (0.03 sec)

mysql> select name,static,urls,lifetime,server from streams;
+--------+--------+----------------------+----------+-----------------+
| name   | static | urls                 | lifetime | server          |
+--------+--------+----------------------+----------+-----------------+
| bunny1 |      1 | file://vod/bunny.mp4 |   108025 | localhost       |
| bunny2 |      1 | file://vod/bunny.mp4 |     3989 | srv02.cdn.local |
+--------+--------+----------------------+----------+-----------------+
2 rows in set (0.00 sec)

Very good, we can create streams from single entry point on all our servers.

Mention that we have explicitly told on what server do we want to create stream. No magic here yet.

Updating Anchor Anchor x2

Updating is more convenient, because you don't need to specify explicitly server:

mysql> update streams set url_prefix='http://srv02.cdn.local' where name='bunny2';
Query OK, 1 row affected (0.02 sec)

mysql> select name,static,urls,lifetime,server,url_prefix from streams;
+--------+--------+----------------------+----------+-----------------+------------------------+
| name   | static | urls                 | lifetime | server          | url_prefix             |
+--------+--------+----------------------+----------+-----------------+------------------------+
| bunny1 |      1 | file://vod/bunny.mp4 |   429009 | localhost       | NULL                   |
| bunny2 |      1 | file://vod/bunny.mp4 |   319980 | srv02.cdn.local | http://srv02.cdn.local |
+--------+--------+----------------------+----------+-----------------+------------------------+
2 rows in set (0.01 sec)

Flussonic knows on what server stream runs, so it will not require from you specifying it. It is very convenient, because you don't need to remember where your stream is.

If you have duplicated stream names on several servers, you still can specify server on which do you want stream update. In the cluster "name" is not a unique thing.

Deletion Anchor Anchor x2

Let's get rid from one of these streams:

mysql> delete from streams where name='bunny1';
Query OK, 1 row affected (0.02 sec)

mysql> select name,static,urls,lifetime,server,url_prefix from streams;
+--------+--------+----------------------+----------+-----------------+------------------------+
| name   | static | urls                 | lifetime | server          | url_prefix             |
+--------+--------+----------------------+----------+-----------------+------------------------+
| bunny2 |      1 | file://vod/bunny.mp4 |   593231 | srv02.cdn.local | http://srv02.cdn.local |
+--------+--------+----------------------+----------+-----------------+------------------------+
1 row in set (0.01 sec)

and total cleanup:

mysql> delete from streams where name='bunny2';
Query OK, 1 row affected (0.02 sec)

mysql> select name,static,urls,lifetime,server,url_prefix from streams;
Empty set (0.00 sec)

Flussonic follows the same rule for deletion as for updating or selecting. If you have single stream in cluster, it will find it and modify config of that remote server.

Migration

Here where some magic begins. If you have configured stream on one server, it is rather easy to migrate it to another server:

mysql> insert into streams (name,urls,dvr,server) values  ('bunny2','file://vod/bunny.mp4','movies 2d 20G','srv02.cdn.local');
Query OK, 1 row affected (0.03 sec)

mysql> select name,static,urls,lifetime,server from streams;
+--------+--------+----------------------+----------+-----------------+
| name   | static | urls                 | lifetime | server          |
+--------+--------+----------------------+----------+-----------------+
| bunny2 |      1 | file://vod/bunny.mp4 |     3889 | srv02.cdn.local |
+--------+--------+----------------------+----------+-----------------+
1 rows in set (0.00 sec)

mysql> update streams set server='srv03.cdn.local' where name='bunny2';
Query OK, 1 row affected (0.02 sec)

mysql> select name,static,urls,lifetime,server from streams;
+--------+--------+----------------------+----------+-----------------+
| name   | static | urls                 | lifetime | server          |
+--------+--------+----------------------+----------+-----------------+
| bunny2 |      1 | file://vod/bunny.mp4 |     3989 | srv03.cdn.local |
+--------+--------+----------------------+----------+-----------------+
1 rows in set (0.00 sec)

You can move streams from any server to any other, flussonic will create them on new server and delete on old.