Skip to content

Flussonic HTTP API

Table of contents:

General information about the API design

We at "Erlyvideo" company provide various products and services, such as Media Server, Watcher, Iris, Retroview and so on.

This page provides up-to-date information on the HTTP API management principles concerning our systems.

Principles of API design

The Flussonic HTTP API is designed for other applications to access our systems and interact with them. Here are the core principles that we try to follow:

  • focus on industry standards and generally accepted practices while maintaining the principle of reasonableness;
  • organizing the API around REST + JSON;
  • using OpenAPI 3.0 format (formerly Swagger) for a machine-readable API description;
  • convenience and simplicity in creating both simple and complex requests (for example, a simple request for streams list and a complex request with multiple filters);
  • consistency in terminology among the systems (the name of an entity in one system must be the same among all the others);
  • standardized access methods to different systems. There might be a collection of one authorization backend on the server and millions of session records in the browsing history storage service. Standardized API should provide access to such systems;
  • preference for HTTP. Access to data on WebSockets is poorly standardized and hard to monitor;
  • obtaining a reasonably limited amount of data for each sample. So the client will not receive a big amount of records by default;
  • supporting the work with a dynamic data type;
  • support of GET requests and declared parameters in the query string to create simple access requests to large collections;
  • idempotent API, i. e. running the same request multiple times produces the same result;
  • supporting work with complex objects, nested subobjects, and collections.

Authentication and authorization

With Flussonic API, you can retrieve data and manage certain Flussonic features over HTTP.

To enable the protection for data retrieval requests use the view_auth user password; directive, and for modification of a state and settings requests with the edit_auth user password; directive in the configuration file /etc/flussonic/flussonic.conf.

For more information about the authorization in Flussonic see: Authorization.

To get an access to Flussonic HTTP API functionality with enabled authentication use a login and password according to HTTP Basic Auth.

Authorization for the edit_auth user password; looks as follows:


GET /flussonic/api/v3/streams HTTP/1.1
Host: FLUSSONIC-IP
Authorization: Basic dXNlcjpwYXNzd29yZA==

HTTP 200 OK
Date: Sun, 19 Sep 2021 19:40:22 GMT
Content-Type: application/json
...

If you need to use Bearer auth, use:


GET /flussonic/api/v3/streams HTTP/1.1
Host: FLUSSONIC-IP
Authorization: Bearer dXNlcjpwYXNzd29yZA==

HTTP 200 OK
Date: Sun, 19 Sep 2021 19:40:22 GMT
Content-Type: application/json
...

Note

Authorization used in other systems may differ, but, in most cases, Bearer auth is used.

OpenAPI Support

Systems that support the principles mentioned in this document provide service descriptions according to the machine-readable OpenAPI 3.0 specification, which evolved from Swagger и JSON Schema.

OpenAPI defines a standard, language-agnostic interface, which allows formalizing a list of service methods and the input and output data formats.

To retrieve the Flussonic API schema, use the following command provided that your Flussonic server is up and running:


curl http://FLUSSONIC-IP:8080/flussonic/api/v3/schema

The example of code below shows the way you can access a list of streams with React Query and openapi-client-axios:


import React from 'react';
import { useQuery } from 'react-query'
import OpenAPIClientAxios from 'openapi-client-axios';


const api = new OpenAPIClientAxios({
  definition: 'http://localhost:8080/flussonic/api/v3/schema',
  axiosConfigDefaults: {
    headers: {
      'Authorization': 'Basic dXNlcjpwYXNzd29yZA==',
    },
  },  
});


function Streams() {
  const { isLoading, error, data } = useQuery({
    queryKey: "streams", 
    queryFn : () => {
      return api.init()
        .then(client => client.streams_list({}))
        .then(res => res.data);
    },
    keepPreviousData: true,
    refetchInterval: refetchInterval,
  });

  if(isLoading) return <div>Loading</div>;
  return <div>
    {data.streams.map((stream) => <div key={stream.name}>{stream.name}</div>}
  </div>;
}

The program above reads the API schema, retrieves a list of endpoints and creates a set of functions from it.

In the example above, streams_list is taken from the schema and created programmatically.

It is to mention that OpenAPI does not allow you to distinguish private and public API.

There is a rule concerning private and public API: if a field has no description (description), it is part of the private API. Thus, it (the field) may change without any warning and backward compatibility.

Swagger UI

For the developer's convenience, there is a built-in Swagger UI:


http://FLUSSONIC-IP:8080/flu/swagger/index.html

To start working with it, you should log in to your Admin UI with your login and password.

Collection sampling

Almost all objects are organized in collections (similar to SQL tables). The network's access to the system to read the data is mostly determined by the way objects are read from the collection.

For the fast-running user interfaces and predictably running programs, it is necessary to:

  • be able to get the minimum required data set,

  • be able to get all the data from the collection with certain reliability.

These two requirements are conflicting. The requirement to limit the data set implies getting only a certain number of objects from the collection, for example, 50 streams out of 2000 on the server.

When the client needs to get the entire list of streams, he will have to make several requests to the server. There is a chance that when new streams appear during the two consecutive requests, these new streams will be missed in the selection.

Note

We do not offer a general approach to capturing snapshots of the collection and allow a certain amount of risk of losing several records when performing page-oriented sampling from a dynamically changing collection.

For predictable access to a subset of the collection, we describe and implement a language that defines the following actions on the collection:

  • filtering (similar to SQL WHERE),

  • sorting (similar to SQL ORDER BY),

  • limiting the number of elements (similar to SQL LIMIT),

  • additional cursor filtering (similar to SQL OFFSET),

  • limiting the number of fields (similar to SQL SELECT)

HTTP methods for accessing collections

The standard way to request a reading from a client to a server is using the query string language and the 'GET' method. It is considered that access that does not involve any modifications is carried out using a GET request (including caching).

Note

The caching is outdated and irrelevant in its original form, but we have implemented it for a convenient start.

When using a query string language, there are two approaches: to use a standard key=value approach or use non-standard delimiters. The second case may cause a number of challenges:

  1. Too complex queries with nested conditions. Writing such conditions in a query string will look extremely clumsy and heavy, which contradicts our core principle — the convenience and simplicity of creating requests.

  2. Incorrect query processing when using additional separators in subsets. In this case, it is necessary to remember the set of possible characters. For example, comma (",") or hyphen ("-") are not used in field names and are not processed in the query string. The ampersand ("&"), in contrast, is a special character for the query string and, if it is not escaped, issues may arise when the request is to be processed.

For example, some companies use the characters - and + in the sorting request to indicate the sorting direction. The + character is a special character and is treated as space when decoding a query string. Therefore, you either need to escape it using %2B, or use a non-standard parser to avoid possible issues with query execution. The use of non-standard parsers makes it difficult for standard libraries to create a request.

Note

There is a feature related to the use of Unicode characters. Some companies encode the SQL query such as WHERE age > 20 in the query string as age>20. Using a Unicode character instead of the standard ASCII character > helps to avoid escaping to place into HTML document. However, it is challenging to type such a request from the keyboard. We refused to use Unicode characters, so it will not be possible to type them from the terminal.

The problems described above intend to illustrate the complexity of using a variety of conditions and versions of the SQL language into a limited HTTP query string language (query string language).

A more complex option is to transmit the request language in JSON format in the body of the POST request. This approach looks like an inevitable solution when using complex nested conditions.

Response structure

When accessing the collection Flussonic returns a JSON-encoded response with the following fields:


{
  "ITEMS": [...],
  "next": ...,
  "prev": ...,
  "estimated_count": ...,
  "timing": ...
}

ITEMS field is replaced with the name of the requested collection, i. e. streams for streams and sessions for sessions.

  • next and prev are the cursor values (the next and the previous, respectively).

  • estimated_count is an approximate number of elements in a collection.

  • timing is a service object with various access times. It is not described as it may be changed or deleted in the future.

Filtering collections

To retrieve filtered responses, add the parameters in a URL query string.

To filter by value (provider), run the following command:


curl http://FLUSSONIC-IP:8080/flussonic/api/streams?provider=Sky

To check if elements are in the list:

provider=Sky,Canal,CNN (values are separated by a comma)

To impose a condition on the nested object field:

stats.alive=true

Field values are compared using the suffixes like _lt. So WHERE stats.delay < 5000 turns into stats.delay_lt=5000.

We have defined a fixed set of suffixes, providing two non-overlapping sets with the existing set of field names among all our systems. It allows us to provide features to create access requests.

Note

For instance, the developers of one of the API variants decided to make not by direct comparison requests using .: delay.lt=5000. However, in this case, they cannot provide access to the fields of nested objects.

Here is a list of supported suffixes:

parameter in a query string SQL
age_lt=50 age < 50
age_lte=50 age <= 50
age_gt=50 age > 50
age_gte=50 age >= 50
age_is=null age IS NULL
age_is_not=null age IS NOT NULL
age_like=pattern age LIKE 'pattern%'

Several filters specified in the query string are applied sequentially to the collection, similar to AND queries in SQL:


curl  http://FLUSSONIC-IP:8080/flussonic/api/v3/streams?stats.bitrate_gt=4000&stats.clients_count_lt=10

Note

We do not offer an OR version for query string due to the complex notation of parentheses.

Note

The filtering fields are specified without a prefix. For example, you could use them like the following filter.age=50. It may be more suitable for programming, validation, etc., but not for a person who will use such an API.

Sorting collections

Collection sorting is needed for displaying in the web interface Flussonic UI and retrieving a predictable page-oriented selection from the collection.

The sorting parameters passed separated by a comma to the key:

sort=stats.ts_delay,-stats.bitrate

By default, ascending-order sort is used. To change the sorting direction, specify - before the field name.

The absence of a prefix before the filtering fields means that we do not allow sort fields in objects in our API.

We added an implicit sorting everywhere in our API. For example, if sorting by the provider field (i. e. a non-unique field) is applied, then several groups of objects will be returned incomprehensibly sorted. To avoid such a situation, we add fields like name and position to the end of the sorting request, applying implicit sorting.

If they are specified explicitly in the list of sorting fields, then re-sorting by them is no longer performed.

We do not specify a list of specific fields for implicit sorting since it may change at any time if we decide to use it differently for a particular case.

Limited collections access (cursors)

If a collection is large, HTTP API should be able to return them part by part.

To get the first 100 elements of a collection, use the limit=100 parameter in the query string.

The question is how to retrieve the next 100 elements.

Note

A solution for SQL databases is to pass the offset field. However, we refused to use such a method because it is computationally expensive (due to the Schlemiel the Painter's algorithm), and it is of no practical use. As we have already mentioned, the collections data may change between the consecutive requests, which means that the offset parameter will return the records not sequentially, but rather in an unknown order without any chance to determine what was lost.

Note

With page-oriented access, you do not need to get the data with an offset of 100. It is necessary to get the next batch of data. The approach with cursors is unusual for the MySQL due to their practical absence in this database. However, in older databases cursors are still in use.

In response to the request for the first 100 elements, the next field (and in the next samples prev) is returned. This field value should be passed in the cursor= parameter in the query string to fetch the following:


$ curl -sS "http://FLUSSONIC-IP:8080/flussonic/api/v3/streams?select=name&limit=1&name_like=a&sort=name"  |jq 
{
  "estimated_count": 5,
  "next": "JTI0cG9zaXRpb25fZ3Q9MiZuYW1lX2d0PWEx",
  "prev": null,
  "streams": [
    {
      "effective": {
        "name": "a1",
        "position": 2,
        "section": "stream",
        "static": true
      },
      "name": "a1"
    }
  ],
  "timing": {
    "filter": 0,
    "limit": 0,
    "load": 3,
    "select": 0,
    "sort": 0
  }
}



$ curl -sS "http://FLUSSONIC-IP:8080/flussonic/api/v3/streams?select=name&limit=1&name_like=a&sort=name&cursor=JTI0cG9zaXRpb25fZ3Q9MiZuYW1lX2d0PWEx"  |jq 
{
  "estimated_count": 5,
  "next": "JTI0cG9zaXRpb25fZ3Q9MyZuYW1lX2d0PWEy",
  "prev": "JTI0cG9zaXRpb25fbHQ9MyZuYW1lX2x0PWEyJiUyNHJldmVyc2VkPXRydWU=",
  "streams": [
    {
      "effective": {
        "name": "a2",
        "position": 3,
        "section": "stream",
        "static": true
      },
      "name": "a2"
    }
  ],
  "timing": {
    "filter": 0,
    "limit": 0,
    "load": 1,
    "select": 0,
    "sort": 0
  }
}

Using consecutive queries, we get all the elements of the data selection. The cursor is arranged quite simple: the values of the sorting fields for the last element of the selection are encoded in it, and an additional filtering is performed before the data is returned.

Limiting the number of returned fields

We provide the possibility to return only a limited number of fields. The total number of fields in the stream data of the media server exceeds 100, and if you need to get only the stream names, then it is rather impractical to request all of the information.

To return only a limited number of fields, specify the select=name,title option. You can also request the fields of nested objects: select=name,status.media_info.

Creating and updating (upsert)

The REST concept dictates the distinction between creating and updating methods for already existing objects.

Note

We have practically abandoned this separation and prefer a more robust option with simultaneous creation or updating of objects. If the methods for creating and updating are separated, the executed code has to have a condition expression with repetitions and perform a loop check. We made it so that, firstly, an object is created on the server. If we get a response that such an object already exists, then try to update it (or not), depending on the situation.

Note

If the idempotency token is not implemented on the server and the client, i. e. unique action ID, then repeated requests on object creation can lead to the creation of multiple objects. The client may not even be aware of this. If there is some network interruption there might not even be any response for a request. In case of the data that we operate with, the object ID is created on the client's side, which means thatthere is no need to ask for the ID generation on the server's side (in contrast to the blind following of REST).

Make a PUT request:


curl -X PUT -H "Content-Type: application/json" -d '{"title":"ORT"}' "http://FLUSSONIC-IP:8080/flussonic/api/v3/streams/ort"

It is very convenient for the API that the object ID occupies only one segment. It means that if, for example, the stream name is a compound name (like sports/football) then you should escape the / character using %2F to access it over API:


curl -X PUT -H "Content-Type: application/json" -d '{"key":"value"}' "http://FLUSSONIC-IP:8080/flussonic/api/v3/streams/sports%2Ffootball"

We call this approach UPSERT because it is similar to SQL UPSERT.

Update protocol

In Media Server, a protocol for updating complex nested objects or even lists of objects is supported.

For example, a stream has a list of sources. How to encode a modification of a particular source or an addition to the list of stream sources in the JSON file sent by a client?

The traditional REST answer is to make a separate endpoint:


/flussonic/api/v3/streams/ort/inputs/4

We preferred to create the rules by which commands for editing elements of a nested list can be encoded in the transmitted JSON file.

We distinguish two types of nested lists: ones that have primary key and the others that do not. For instance, stream source does not have a primary key, i. e. it does not have a unique field due to the nature of the data type. However, MPTS programs have such primary key that is called service id.

Partial editing is supported for the nested lists. If an update request is received, and the object already exists, and the client passes a list in this object, it is not overwritten, but edited.

If a nested list has a primary key, it is used as an ID for subobjects when editing. If a nested list does not have a primary key, the special $index field is being used. A new object is created if this field is not passed.

Example:


$ curl -sS -d '{"inputs": [{"url": "udp://239.0.0.1:1234"}]}' -X PUT  "http://FLUSSONIC-IP:8080/flussonic/api/v3/streams/ort"  |jq 
{
  "name": "ort",
  "named_by": "config",
  "position": 7,
  "static": true,
  "inputs": [
    {
      "url": "udp://239.0.0.1:1234"
    }
  ]
}

To edit an existing source, use the PUT HTTP method.

Example:


$ curl -sS -d '{"inputs": [{"$index": 0, "url": "udp://239.0.0.2:1234"}]}' -X PUT "http://FLUSSONIC-IP:8080/flussonic/api/v3/streams/ort"  |jq 
{
  "inputs": [
    {
      "url": "udp://239.0.0.2:1234"
    }
  ],
  "name": "ort",
  "named_by": "config"
}
...

To remove an existing source, specify a $delete field in the nested object.

Example:


$ curl -sS -d '{"inputs": [{"$index": 0, "$delete": true}]}' -X PUT  "http://FLUSSONIC-IP:8080/flussonic/api/v3/streams/ort"  |jq 
{
  "name": "ort",
  "named_by": "config",
  "position": 7,
  "static": true,
}

Reading objects

To read an object use a GET HTTP method:


curl -sS "http://FLUSSONIC-IP:8080/flussonic/api/v3/streams/ort"  |jq 

Response: JSON-encoded file with the information about the object.


{
  "effective": {
    "name": "ort",
    "position": 7,
    "section": "stream",
    "static": true
  },
  "name": "ort",
  "named_by": "config",
  "position": 7,
  "static": true,
  "stats": {
    "alive": false,
    "bytes_in": 0,
    "bytes_out": 0,
    "client_count": 0,
    "dvr_enabled": false,
    "dvr_only": false,
    "dvr_replication_running": false,
    "id": "61496e6e-a22f-46af-bce5-5479f9067ead",
    "input_error_rate": 0,
    "last_access_at": 1632202350648,
    "lifetime": 0,
    "opened_at": 1632202350648,
    "out_bandwidth": 0,
    "publish_enabled": false,
    "remote": false,
    "retry_count": 19,
    "running": true,
    "running_transcoder": false,
    "start_running_at": 1632202350648,
    "transcoder_overloaded": false
  }
}

Removing objects

To delete an object, use a DELETE HTTP method:


curl -sS -X DELETE "http://FLUSSONIC-IP:8080/flussonic/api/v3/streams/ort"

Response: HTTP 204 with empty body response.

Managing the streams in Media Server

Flussonic Media Server allows you to perform some actions on the streams:

  • get a list of streams:

curl -sS "http://FLUSSONIC-IP:8080/flussonic/api/v3/streams" |jq

Response: JSON-encoded file with the information about the list of streams.


{
  "estimated_count": 13,
  "next": null,
  "prev": null,
  "streams": [
    {
      "inputs": [
        {
          "url": "fake://fake"
        }
      ],
      "name": "example_stream",
      "named_by": "config",
      "position": 0,
      "static": true,
      "stats": {
        "alive": true,
        "bitrate": 187,
        "bytes_in": 6105063537,
        "bytes_out": 0,
        "client_count": 0,
        "dvr_enabled": false,
        "dvr_only": false,
        "dvr_replication_running": false,
        "id": "6149e3a0-d8ec-4edf-8f92-463e85fa8ad2",
        "input_bitrate": 187,
        "input_error_rate": 0,
        "last_access_at": 1632232352867,
        "last_dts": 1632394204986.6667,
        "last_dts_at": 1632394205491,
        "lifetime": 161851626.66674805,
...

  • create a stream:

curl -X PUT -H "Content-Type: application/json" -d '{"inputs":[{"url":"fake://fake"}],"title":"Example"}' "http://FLUSSONIC-IP:8080/flussonic/api/v3/streams/STREAM_NAME" |jq

Response: JSON-encoded file with the information about the created stream.


{
  "inputs": [
    {
      "url": "fake://fake"
    }
  ],
  "name": "new_stream",
  "named_by": "config",
  "position": 14,
  "static": true,
  "stats": {
    "alive": true,
    "bitrate": 187,
    "bytes_in": 378985,
    "bytes_out": 0,
    "client_count": 0,
    "dvr_enabled": false,
    "dvr_only": false,
    "dvr_replication_running": false,
    "id": "614c627d-4f43-45a3-a11d-674bec71d59d",
    "input_bitrate": 187,
    "input_error_rate": 0,
    "last_access_at": 1632395901317,
    "last_dts": 1632395911986.6667,
    "last_dts_at": 1632395912497,
    "lifetime": 10666.666748046875,
    "media_info": {
      "title": "Example",
      "tracks": [
        {
          "bframes": 0,
          "bitrate": 161,
          "codec": "h264",
          "content": "video",
          "fps": 25,
          "gop_size": 25,
          "height": 240,
          "last_gop": 25,
          "level": "2.1",
          "pix_fmt": "yuv420p",
          "pixel_height": 240,
          "pixel_width": 320,
          "profile": "Baseline",
          "sar_height": 1,
          "sar_width": 1,
          "track_id": "v1",
          "width": 320
        },
        {
          "bitrate": 26,
          "channels": 2,
          "codec": "aac",
          "content": "audio",
          "lang": "eng",
          "sample_rate": 48000,
          "track_id": "a1"
        }
      ]
    },
    "opened_at": 1632395901317,
    "out_bandwidth": 0,
    "output_bitrate": 187,
    "publish_enabled": false,
    "remote": false,
    "retry_count": 0,
    "running": true,
    "running_transcoder": false,
    "source_id": "614c627d-5407-4419-b73b-69a25f4f010a",
    "start_running_at": 1632395901317,
    "transcoder_overloaded": false,
    "ts_delay": 583,
    "url": "fake://fake"
  },
  "title": "Example",
  "urls": [
    {
      "url": "fake://fake"
    }
  ]
}

  • get an information about the stream:

curl -sS "http://FLUSSONIC-IP:8080/flussonic/api/v3/streams/STREAM_NAME" |jq

Response: JSON-encoded file with the information about the requested stream.


{
  "chunk_duration": 300,
  "cmaf_enabled": true,
  "inputs": [
    {
      "url": "fake://fake"
    }
  ],
  "name": "example_stream",
  "named_by": "config",
  "position": 9,
  "static": true,
  "stats": {
    "alive": true,
    "bitrate": 187,
    "bytes_in": 6034345652,
    "bytes_out": 254192,
    "client_count": 0,
    "dvr_enabled": false,
    "dvr_only": false,
    "dvr_replication_running": false,
    "id": "6149e3a0-d8f0-4e84-a0c8-3349ae265f05",
    "input_bitrate": 189,
    "input_error_rate": 0,
    "input_media_info": {
      "tracks": [
        {
          "bframes": 0,
          "bitrate": 163,
          "codec": "h264",
          "content": "video",
          "fps": 25,
          "gop_size": 25,

  • remove a stream:

curl -sS -X DELETE "http://FLUSSONIC-IP:8080/flussonic/api/v3/streams/STREAM_NAME"

Response: empty.

It is important to point out that an active stream is an object with dynamic characteristics, i .e. characteristics that change rapidly with time. Thus, when reading the stream's data, you may get the data different from the one written initially, unlike the SQL database. For instance, the runtime field is calculated when the stream is running and cannot be written to the server.