Flussonic Media Server documentation

DVB subtitles OCR recognition to WebVTT

Starting with version 4.7.2, Flussonic Media Server now has the ability to recognize DVB subtitles in WebVTT. Recognition is implemented using the Tesseract program.

DVB subtitles Anchor Anchor x2

The DVB standard (Digital Video Broadcast) defines a bitmap based subtitling format.

In the DVB stream from the MPEG-TS satellite, both the text and the picture can be subtitled. Most often it's a picture because it's so much safer for viewing on different devices that may not have the right fonts for drawing text.

For example:

ffmpeg -i video.ts 
Stream #0:0[0x1a4]: Video: h264 (High) ([27][0][0][0] / 0x001B), yuv420p(tv, bt709), 1920x1080 [SAR 1:1 DAR 16:9], 25 fps, 25 tbr, 90k tbn, 50 tbc
Stream #0:1[0x1ae](fra): Audio: eac3 ([6][0][0][0] / 0x0006), 48000 Hz, stereo, fltp, 128 kb/s
Stream #0:2[0x1af](qad): Audio: eac3 ([6][0][0][0] / 0x0006), 48000 Hz, stereo, fltp, 128 kb/s
Stream #0:3[0x1b8](fra): Subtitle: dvb_subtitle ([6][0][0][0] / 0x0006) (hearing impaired)
Stream #0:4[0x1b0](qaa): Audio: eac3 ([6][0][0][0] / 0x0006), 48000 Hz, stereo, fltp, 128 kb/s
Stream #0:5[0x1b9](fra): Subtitle: dvb_subtitle ([6][0][0][0] / 0x0006)

Here dvb_subtitle are subtitles that come in images.

Such subtitles can show video-consoles, some TVs, a VLC player, but not an iPhone / Android.

Flussonic Media Server can convert pictures back to the original text in order to be able to show subtitles on mobile devices and players without DVB support.

WebVTT subtitles Anchor Anchor x2

WebVTT (Web Video Text Tracks Format) is a common subtitle format that is supported by browsers and provides other options. Recognizing DVB subtitles in WebVTT, you can reduce the load on the transmission of the stream.

The WebVTT format file is a regular text file with the extension .vtt, in which in the form of a line-by-line list labels with start time and end time and text messages for output to these labels are registered. To the video container, you can connect several WebVTT files with text in different languages. For each language you need to create a separate file.

These files can be used to transfer additional data to JS players. For example, URL image preview for video frames. The WebVTT standard also supports CSS styling and placement options in the video preview area.

Installation Anchor Anchor x2

Tesseract — high-quality console OCR engine with open source. Currently, the program works with UTF-8, and language support (including Russian version 3.0) by means of additional modules.

For Ubuntu, you can install the package flussonic-tesseract:

apt install flussonic-tesseract

After installing Tesseract, enable the options for the stream:

allow_subtitles=true dvbsubs_ocr=true;

Example:

stream ort {
  url tshttp://source:8080/stream allow_subtitles=true dvbsubs_ocr=true;
}

Do not forget to apply new configuration via command: /etc/init.d/flussonic reload

Successful Tesseract start will cause the following entries appear in the logs:

09:44:17.986 <0.966.0> [sow] tesseract_worker:58 start ocr for slv
09:44:18.275 <0.966.0> [sow] tesseract_worker:58 start ocr for srp
09:44:18.759 <0.966.0> [sow] tesseract_worker:58 start ocr for swe
09:44:19.045 <0.966.0> [sow] tesseract_worker:58 start ocr for dan
09:44:19.328 <0.966.0> [sow] tesseract_worker:58 start ocr for nor

Example index.m3u8 with subtitles:

#EXTM3U
#EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID="subs",NAME="English",DEFAULT=YES,AUTOSELECT=YES,FORCED=NO,LANGUAGE="eng",URI="http://flussonic-ip/index.m3u8"
#EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID="subs",NAME="French",DEFAULT=NO,AUTOSELECT=YES,FORCED=NO,LANGUAGE="fra",URI="http://flussonic-ip/index.m3u8"
#EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID="subs",NAME="German",DEFAULT=NO,AUTOSELECT=YES,FORCED=NO,LANGUAGE="deu",URI="http://flussonic-ip/index.m3u8"
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=450560,RESOLUTION=480x352,SUBTITLES="subs"
http://flussonic-ip/index.m3u8
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=855040,RESOLUTION=480x352,SUBTITLES="subs"
http://flussonic-ip/index.m3u8