Inference server detects and tracks objects of interest (people, faces, vehicles) on video and forms episodes. See Flussonic Vision Identification Server API for more information regarding face identification and persons.

Inference server obtains the list of streams to be processed from the external configuration backend API. In most cases this external configuration is provided by Flussonic Central which helds the entire cluster configuration.

Inference server captures and decodes configured streams to get raw, uncompressed frames. Because processing is computationally expensive, it cannot handle every frame from every configured stream. Instead, the Inference server processes the most recent frame from each stream after completing the previous processing.

The result of processing of a frame is detection. It contains confidence of the detection, coordinates of the object within the frame, object's class (e.g., face, vehicle). After detection, the system recognizes the object: for example, by calculating a digital fingerprint for a face or extracting the text of a license plate. Those attributes help identify whether a sequence of detections corresponds to the same person or the same vehicle.

From those sequences episodes are formed. An episode characterizes the continuous presence of a single object in the video. It has a start timestamp (when this object first appeared) and end timestamp (when this object leaves the scene). An episode is called open if the object is still in the scene.

A list of episodes is available via episodes_list API operation. It supports long polling when the corresponding parameter in the query is set. Clients connected using long polling receive each update for open episodes.

Whenever the episode's attributes are updated, server adjusts the updated_at field of the episode. This can be used to request recently updated episodes only by specifying updated_at_gt=<sometimestamp> query parameter for the episodes_list operation

Key Terms

• Detection: finds objects in the video frame, determines their coordinates and object type (e.g., face, license plate, person).
• License plate recognition recognizes the characters of the license plate.
• Face Recognition: generates digital representation of the face, known as digital fingerprints.
• Episode represents the continuous tracking of a single object (like a person or a vehicle) in the video.
• Face identification matches the recognized face with known persons.
  See Flussonic Vision Identification Server API for more information regarding face identification and persons

Episode

An episode characterizes the continuous presence of a single object in the video and is described by the following attributes:

• start time - when the object first appeared
• end time - when the object disappeared
• detection data - the object class, confidence level, keypoints, coordinates in the frame, etc.
• recognition data - a string containing characters of the detected license plate or text representation of digital fingerprints for faces
• identification data - identifiers of persons matched with the recognized face in the episode.

Episode's Lifecycle

The episode's lifecycle is defined by three states: "opened", "started", "closed".

• opened - a new object has just appeared in the frame, but there's not yet enough confidence to rule out a false detection
• started - sufficient confidence in the object's detection (typically, confident detection and recognition over a sequence of frames)
• closed - the object is no longer in the frame, and its continuous presence tracking has ended

Streams information.
This API provides readonly list of streams being processed by the server.
Each record in the list includes some useful metrics about stream's health and videoanalytics status.

Configuration of streams, their input URLs and what videoanalytics should be used on each of them is set by providing external configuration backend's URL to the service. This URL should be provided in the service's configuration file as CONFIG_EXTERNAL parameter.
See Vision Configuration Backend API for more information.

Service configuration

Image analytics

Episodes

Server metrics and status

Counters provides historical data about changing the number of objects of interest in form of records.
Each record represents statistical information such as how many new visitors arrived into the area within a timeframe, what was the maximum and minimum number of visitors in the area, etc.