Software IntegrationGStreamer

GStreamer - ZED Data mux

Open in ClaudeOpen in ChatGPT

The ZED Data Mux, zeddatamux GStreamer element, allows adding ZED Metadata to a ZED video stream. This is useful if the metadata is removed by a transform filter element that does not automatically propagate it.

Properties

No properties

Example pipelines

Inject ZED metadata information in a stream

Example pipeline that resizes the 2K RGB and depth streams to VGA resolution for display purposes, remuxing the Object Detection metadata as input for the zedodoverlay element:

$gst-launch-1.0 zeddatamux name=mux \
> zedsrc stream-type=4 camera-resolution=0 camera-fps=15 od-detection-model=0 od-enabled=true ! \
> zeddemux stream-data=true is-depth=true name=demux \
> demux.src_aux ! queue ! autovideoconvert ! videoscale ! video/x-raw,width=672,height=376 ! queue ! fpsdisplaysink \
> demux.src_data ! mux.sink_data \
> demux.src_left ! queue ! videoscale ! video/x-raw,width=672,height=376 ! mux.sink_video \
> mux.src ! queue ! zedodoverlay ! queue ! \
> autovideoconvert ! fpsdisplaysink

image

  • Line 1: a zeddatamux element is defined with the name mux.
  • Line 2: the zedsrc element is configured to acquire Left and Depth data with HD2K resolution at 30 FPS, enabling Object Detection with the MULTI-CLASS model.
  • Line 3: a zeddemux element is defined to split the pipeline into 3 separate branches: one to process the Left Color stream, one to process the Depth stream, and one with binary data created from the Sensors and Detected Objects metadata, which is injected into the mux element.
  • Line 4: the depth stream is processed by a videoscale transform filter element to reduce the size of the image and is finally displayed on the screen with FPS info.
  • Line 5: the left stream is processed by a videoscale transform filter element to reduce the size of the image and is injected into the mux element.
  • Line 6: the muxed stream from the mux element is processed by the zedodoverlay transform filter element to draw object detection data.
  • Line 7: the object detection result is displayed on screen with FPS info.

The zeddatamux element is important to ensure that the stream filtered by zedodoverlay contains the Object Detection metadata, which could be removed from the original ZED stream by the videoscale transform filter if it does not handle this kind of metadata.

The ZED metadata is designed to handle scaling transformations of the image stream; in this particular case, the object bounding boxes are rendered correctly even if the filtered images have a size different from the grabbed frames.