GStreamer - ZED Data mux

The ZED Data Mux, zeddatamux GStreamer element, allows adding ZED Metadata to a ZED video stream. This is useful if the metadata is removed by a transform filter element that does not automatically propagate it.

The zeddatamux element has the following pads:

sink_video: sink pad receiving a video stream compatible with the ZED caps
sink_data: sink pad receiving the application/data metadata stream generated by the src_data pad of a zeddemux element
src: source pad pushing the video stream with the metadata attached

Properties

The element defines no specific property. gst-inspect-1.0 zeddatamux only reports the name and parent properties inherited from GstObject:

  name                : The name of the object
                        flags: readable, writable, 0x2000
                        String. Default: "zeddatamux0"
  parent              : The parent of the object
                        flags: readable, writable, 0x2000
                        Object of type "GstObject"

Example pipelines

Inject ZED metadata information in a stream

Example pipeline that resizes the 2K RGB and depth streams to VGA resolution for display purposes, remuxing the Object Detection metadata as input for the zedodoverlay element:

gst-launch-1.0 zeddatamux name=mux \
 zedsrc stream-type=4 camera-resolution=0 camera-fps=15 od-detection-model=0 od-enabled=true ! \
 zeddemux stream-data=true is-depth=true name=demux \
 demux.src_aux ! queue ! autovideoconvert ! videoscale ! video/x-raw,width=672,height=376 ! queue ! fpsdisplaysink \
 demux.src_data ! mux.sink_data \
 demux.src_left ! queue ! videoscale ! video/x-raw,width=672,height=376 ! mux.sink_video \
 mux.src ! queue ! zedodoverlay ! queue ! \
 autovideoconvert ! fpsdisplaysink

Pipeline graph: zeddemux splits the stream into a rescaled depth display branch, a rescaled left branch and a metadata branch, both recombined by zeddatamux before zedodoverlay

Line 1: a zeddatamux element is defined with the name mux.
Line 2: the zedsrc element is configured to acquire Left and Depth data with HD2K resolution at 15 FPS, enabling Object Detection with the MULTI-CLASS FAST model.
Line 3: a zeddemux element is defined to split the pipeline into 3 separate branches: one to process the Left Color stream, one to process the Depth stream, and one with binary data created from the Sensors and Detected Objects metadata, which is injected into the mux element.
Line 4: the depth stream is processed by a videoscale transform filter element to reduce the size of the image and is finally displayed on the screen with FPS info.
Line 5: the metadata stream from the src_data pad of demux is injected into the sink_data pad of the mux element.
Line 6: the left stream is processed by a videoscale transform filter element to reduce the size of the image and is injected into the sink_video pad of the mux element.
Line 7: the muxed stream from the mux element is processed by the zedodoverlay transform filter element to draw object detection data.
Line 8: the object detection result is converted and displayed on screen with FPS info.

The zeddatamux element is important to ensure that the stream filtered by zedodoverlay contains the Object Detection metadata, which could be removed from the original ZED stream by the videoscale transform filter if it does not handle this kind of metadata.

The ZED metadata is designed to handle scaling transformations of the image stream; in this particular case, the object bounding boxes are rendered correctly even if the filtered images have a size different from the grabbed frames.