Getting Started with GStreamer and ZED

GStreamer is a popular framework used to create custom media pipelines by combining modular plugins. The ZED GStreamer plugins greatly simplify the use of the ZED camera and SDK in a GStreamer media pipeline.

The ZED GStreamer package is composed of five plugins providing five elements, a metadata library, and a Linux RTSP Server application.

Plugins

Name	Classification	Source Code	Description
zedsrc	Source	GitHub	acquires the camera color image, depth map, sensor data, and object detection information and pushes them into a GStreamer pipeline.
zedxonesrc^*	Source	GitHub	acquires ZED X One monocular camera color images and pushes them into a GStreamer pipeline.
zeddemux	Stream Demuxer	GitHub	receives a composite `zedsrc` stream (`color left + color right + metadata` or `color left + depth map + metadata`), processes the depth data when present, and pushes the result into two separate new streams named `src_left` and `src_aux`. A third source pad is optionally created for metadata to be processed externally.
zeddatamux	Metadata Muxer	GitHub	receives a video stream compatible with ZED caps and a ZED Data Stream generated by `zeddemux`, then adds the metadata to the video stream. This is useful if the metadata is removed by a filter that does not automatically propagate it.
zeddatacsvsink	File Sink	GitHub	sink element that receives ZED metadata, extracts the Positional Tracking and Sensors Data information, and saves it to a CSV file. Demonstrates how to retrieve and parse Positional Tracking and Sensors metadata.
zedodoverlay	Filter Effect Video	GitHub	transform filter element that receives a ZED combined stream with metadata, extracts object detection information, and draws the overlays on the incoming image frames. Demonstrates how to retrieve and parse Object Detection metadata.

^* Only available on NVIDIA® Jetson™ devices with GMSL2 capabilities

Libraries

Name	Classification	Source Code	Description
zedmeta	Metadata Library	GitHub	GStreamer library to define and handle the ZED metadata (Positional Tracking data, Sensors data, Detected Object data, Detected Skeletons data).

Applications

Name	Classification	Source Code	Description
RTSP Server	Server application	GitHub	Linux application that instantiates an RTSP server from a “gst-launch”-style text launch pipeline.

Installation

Prerequisites

Stereo camera: ZED X / ZED X Mini / ZED X Nano, ZED 2i, ZED 2, ZED Mini, ZED X One
CMake (v3.1+)
GStreamer 1.0

Windows installation

Install the latest ZED SDK from the official download page [Optional to compile the zedsrc plugin to acquire data from a ZED camera device]
Install Git
Install CMake
Install a GStreamer distribution (both runtime and development installers).
The installer should set the installation path via the GSTREAMER_1_0_ROOT_X86_64 environment variable.
Add the path %GSTREAMER_1_0_ROOT_X86_64%\bin to the system variable PATH
[Optional] Install OpenCV to build the zedodoverlay filter

Run the following commands from a terminal or command prompt, assuming CMake and Git are in your PATH.

$ git clone https://github.com/stereolabs/zed-gstreamer.git
$ cd zed-gstreamer
$ mkdir build
$ cd build
$ cmake -G "Visual Studio 16 2019" ..
$ cmake --build . --target INSTALL --config Release

Linux installation

Install prerequisites

Install the latest ZED SDK from the official download page
Update list of apt available packages

sudo apt update
Install GCC compiler and build tools

sudo apt install build-essential
Install CMake build system

sudo apt install cmake

Install GStreamer, the development packages and useful tools:

$ sudo apt install \
> libgstreamer1.0-0 \
> gstreamer1.0-libav \
> libgstrtspserver-1.0-0 \
> gstreamer1.0-tools \
> gstreamer1.0-x \
> gstreamer1.0-alsa \
> gstreamer1.0-gl \
> gstreamer1.0-gtk3 \
> gstreamer1.0-qt5 \
> gstreamer1.0-pulseaudio \
> libgstreamer1.0-dev \
> libgstrtspserver-1.0-dev \
> libgstreamer-plugins-base1.0-0 \
> libgstreamer-plugins-base1.0-dev \
> libgstreamer-plugins-good1.0-0 \
> libgstreamer-plugins-good1.0-dev \
> libgstreamer-plugins-bad1.0-0 \
> libgstreamer-plugins-bad1.0-dev

[Optional] Install OpenCV to build the zedodoverlay filter

$ sudo apt install libopencv-dev libopencv-contrib-dev

Clone the repository

$ git clone https://github.com/stereolabs/zed-gstreamer.git
$ cd zed-gstreamer

Build

$ mkdir build
$ cd build
$ cmake -DCMAKE_BUILD_TYPE=Release ..
$ make # Note: do not use the `-j` flag because parallel build is not supported
$ sudo make install

Installation test

Open a command line console and try the following commands to verify that everything is correctly installed:

Check ZED Video Source Plugin installation inspecting its properties:
```
$ gst-inspect-1.0 zedsrc
```
Check ZED X One Video Source Plugin installation inspecting its properties (only Jetson™):
```
$ gst-inspect-1.0 zedxonesrc
```
Check ZED Video Demuxer installation inspecting its properties:
```
$ gst-inspect-1.0 zeddemux
```
Check ZED Data Mux Plugin installation inspecting its properties:
```
$ gst-inspect-1.0 zeddatamux
```
Check ZED CSV Sink Plugin installation inspecting its properties:
```
$ gst-inspect-1.0 zeddatacsvsink
```
Check ZED Object Detection Overlay Plugin installation inspecting its properties:
```
$ gst-inspect-1.0 zedodoverlay
```

Use cases

The following is a list of use case examples, each linked to its relevant documentation page: