> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.stereolabs.com/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.stereolabs.com/_mcp/server.

# GStreamer - ZED Metadata

The stream of data injected in a GStreamer pipeline by the [`zedsrc` element](/docs/integrations/gstreamer/zed-camera-source/) contains color and depth information plus metadata with sensors and object detection data. The [`zedxonesrc` element](/docs/integrations/gstreamer/zed-x-one-camera-source-element) attaches the same metadata structure to its buffers, with the object detection and sensors sections left empty.

The ZED metadata are defined in the GStreamer [library `gstzedmeta`](https://github.com/stereolabs/zed-gstreamer/tree/master/gst-zed-meta).

## Metadata information

* `GstZedSrcMeta`: metadata global container
* `ZedInfo`: general information about the ZED camera that acquired the data.
* `ZedPose`: Positional Tracking information.
* `ZedSensors`: data from the camera sensors (if available).
* `ZedImu`: 6 DOF inertial data (if available).
* `ZedMag`: 3 DOF magnetic field data (if available).
* `ZedEnv`: environmental data, atmospheric pressure and internal temperature (if available).
* `ZedCamTemp`: camera CMOS temperatures (if available).
* `ZedObjectData`: object detection data (if available).

### Data Structures

#### GstZedSrcMeta

**`C++`**

```cpp C++
struct _GstZedSrcMeta {
    GstMeta meta;

    ZedInfo info;

    ZedPose pose;
    ZedSensors sens;

    gboolean od_enabled;
    guint8 obj_count;
    guint64 frame_id;
    ZedObjectData objects[256];
};
```

* `meta`: metadata identifier field.
* `info`: camera information.
* `pose`: camera pose.
* `sens`: sensors data.
* `od_enabled`: indicates if object detection data are available.
* `obj_count`: the number of detected objects \[max 255, since the field is a `guint8`, although the `objects` array has 256 entries].
* `frame_id`: track the meta/buffer throughout the GStreamer pipeline (when working with source code)
* `objects`: array of detected objects.

#### ZedInfo

**`C++`**

```cpp C++
struct _ZedInfo {
    gint cam_model;
    gint stream_type;
    guint grab_single_frame_width;
    guint grab_single_frame_height;
};
```

* `cam_model`: [camera model](https://www.stereolabs.com/docs/api/group__Video__group.html#gaa29851790d3e528f42f539334e6e5887), i.e. the value of the `sl::MODEL` enum of the ZED SDK (`0` : ZED, `1` : ZED Mini, `2` : ZED 2, `3` : ZED 2i, `4` : ZED X, `5` : ZED X Mini, `9` : ZED X Nano, `11` : Virtual ZED X, `30` : ZED X One GS, `31` : ZED X One 4K, `33` : ZED X One Core *\[ZED SDK v5.5 and newer]*). The field is a direct copy of the `sl::MODEL` enum, so the set of possible values follows the ZED SDK release used to build the plugin.
* `stream_type`: type of stream, i.e. the resolved value of the `stream-type` property of the source element (`0` : Left image, `1` : Right image, `2` : Stereo couple up/down, `3` : 16-bit depth map, `4` : Left + Depth up/down, `5` : Stereo couple left/right). On NVIDIA® Jetson™ with ZED SDK v5.2 or newer the zero-copy NV12 values `6` (left), `7` (stereo side-by-side) and `8` (right) can also be reported.
* `grab_single_frame_width`: original width of image data.
* `grab_single_frame_height`: original height of image data.

#### ZedPose

**`C++`**

```cpp C++
struct _ZedPose {
    gboolean pose_avail;
    gint pos_tracking_state;
    gfloat pos[3];
    gfloat orient[3];
};
```

* `pose_avail`: indicates if camera pose is available.
* `pos_tracking_state`: [status of the Positional Tracking algorithm](https://www.stereolabs.com/docs/api/group__PositionalTracking__group.html#gae481924a311113375bb3a99748d79e17), i.e. the value of the `sl::POSITIONAL_TRACKING_STATE` enum of the ZED SDK (`0` : SEARCHING *\[deprecated, no longer returned]*, `1` : OK, `2` : OFF, `3` : FPS\_TOO\_LOW, `4` : SEARCHING\_FLOOR\_PLANE, `5` : UNAVAILABLE)
* `pos`: camera position
* `orient`: camera orientation (Euler angles).

#### ZedSensors

**`C++`**

```cpp C++
struct _ZedSensors {
    gboolean sens_avail;
    ZedImu imu;
    ZedMag mag;
    ZedEnv env;
    ZedCamTemp temp;
};

```

* `sens_avail`: indicates if sensors data are available (all camera models with IMU support, i.e. every model except the original ZED).
* `imu`: IMU data.
* `mag`: Magnetometer data.
* `env`: environment data.
* `temp`: camera temperatures data

> **Note**
>
> An example of how to retrieve Sensors data is provided with the [`gstzeddatacsvsink`](/docs/integrations/gstreamer/zed-data-csv-sink/) [element source code](https://github.com/stereolabs/zed-gstreamer/tree/master/gst-zed-data-csv-sink).

#### ZedImu

**`C++`**

```cpp C++
struct _ZedImu {
    gboolean imu_avail;
    gfloat acc[3];
    gfloat gyro[3];
    gfloat temp;
};
```

* `imu_avail`: indicates if IMU data are available (all camera models with IMU support, i.e. every model except the original ZED).
* `acc`: 3 DOF accelerometer data in \[m/s²].
* `gyro`: 3 DOF gyroscope data in \[rad/sec].
* `temp`: IMU temperature in \[°C].

#### ZedMag

**`C++`**

```cpp C++
struct _ZedMag {
    gboolean mag_avail;
    gfloat mag[3];
};
```

* `mag_avail`: indicates if magnetometer data are available (ZED 2 and ZED 2i only).
* `mag`: 3 DOF magnetic field data in \[µT].

#### ZedEnv

**`C++`**

```cpp C++
struct _ZedEnv {
    gboolean env_avail;
    gfloat press;
    gfloat temp;
};
```

* `env_avail`: indicates if environment data are available (ZED 2 and ZED 2i only).
* `press`: atmospheric pressure in \[hPa].
* `temp`: internal camera temperature in \[°C].

#### ZedCamTemp

**`C++`**

```cpp C++
struct _ZedCamTemp {
    gboolean temp_avail;
    gfloat temp_cam_left;
    gfloat temp_cam_right;
};
```

* `temp_avail`: indicates if CMOS temperatures data are available (ZED 2 and ZED 2i only).
* `temp_cam_left`: temperature of the left CMOS sensor in \[°C].
* `temp_cam_right`: temperature of the right CMOS sensor in \[°C].

#### ZedObjectData

**`C++`**

```cpp C++
struct _ZedObjectData {
    gint id;

    OBJECT_CLASS label;
    OBJECT_SUBCLASS sublabel;
    OBJECT_TRACKING_STATE tracking_state;
    OBJECT_ACTION_STATE action_state;

    gfloat confidence;

    gfloat position[3];
    gfloat position_covariance[6];

    gfloat velocity[3];

    unsigned int bounding_box_2d[4][2];

    /* 3D bounding box of the person represented as eight 3D points
          1 ------ 2
         /        /|
        0 ------ 3 |
        | Object | 6
        |        |/
        4 ------ 7
    */
    gfloat bounding_box_3d[8][3];

    gfloat dimensions[3]; // 3D object dimensions: width, height, length

    gboolean skeletons_avail;

    gint skel_format; // indicates if it's 18, 34, 38, or 70 skeleton model

    gfloat keypoint_2d[70][2]; // Negative coordinates -> point not valid
    gfloat keypoint_3d[70][3]; // Nan coordinates -> point not valid

    gfloat head_bounding_box_2d[4][2];
    gfloat head_bounding_box_3d[8][3];
    gfloat head_position[3];
};
```

* `id`: unique identifier of the tracked object.
* `label`: class of the identified object.
* `sublabel`: subclass of the identified object \[only for MULTICLASS models]
* `tracking_state`: tracking status of the object.
* `action_state`: action status of the object (static or moving).
* `confidence`: confidence level of the detection \[0, 100].
* `position`: 3D position of the center of the object.
* `position_covariance`: covariance matrix of the position.
* `velocity`: velocity of the object
* `bounding_box_2d`: 2D image coordinates of the four corners of the bounding box
* `bounding_box_3d`: 3D world coordinates of the eight corners of the bounding box
* `dimensions`: 3D dimensions of the 3D bounding box
* `skeletons_avail`: indicates if a skeleton tracking detection was enabled and if skeleton data are available
* `skel_format`: number of keypoints of the body format in use (`18`, `34`, `38` or `70`), driven by the `bt-format` property of the `zedsrc` element, which currently exposes the 18, 34 and 38 keypoint formats. It tells how many entries of `keypoint_2d`/`keypoint_3d` are meaningful.
* `keypoint_2d`: 2D image coordinates of the skeleton joints. The array is always dimensioned for the largest body format (70 joints); only the first `skel_format` entries are valid.
* `keypoint_3d`: 3D world coordinates of the skeleton joints, with the same convention as `keypoint_2d`.
* `head_bounding_box_2d`: 2D image coordinates of the four corners of the bounding box of the head
* `head_bounding_box_3d`: 3D world coordinates of the eight corners of the bounding box of the head
* `head_position`: 3D world coordinates of the position of the center of the head

**`C++`**

```cpp C++
enum class OBJECT_CLASS {
    PERSON = 0,          /**< For people detection */
    VEHICLE = 1,         /**< For vehicle detection. It can be cars, trucks, buses, motorcycles etc */
    BAG = 2,             /**< For bag detection (backpack, handbag, suitcase) */
    ANIMAL = 3,          /**< For animal detection (cow, sheep, horse, dog, cat, bird, etc) */
    ELECTRONICS = 4,     /**< For electronic device detection (cellphone, laptop, etc) */
    FRUIT_VEGETABLE = 5, /**< For fruit and vegetable detection (banana, apple, orange, carrot, etc) */
    SPORT = 6,           /**< For sport-related object detection (sportball) */
    LAST
};

enum class OBJECT_SUBCLASS {
    PERSON = 0,       /**< PERSON / PERSON_BODY */
    PERSON_HEAD = 22, /**< PERSON */
    BICYCLE = 1,      /**< VEHICLE */
    CAR = 2,          /**< VEHICLE */
    MOTORBIKE = 3,    /**< VEHICLE */
    BUS = 4,          /**< VEHICLE */
    TRUCK = 5,        /**< VEHICLE */
    BOAT = 6,         /**< VEHICLE */
    BACKPACK = 7,     /**< BAG */
    HANDBAG = 8,      /**< BAG */
    SUITCASE = 9,     /**< BAG */
    BIRD = 10,        /**< ANIMAL */
    CAT = 11,         /**< ANIMAL */
    DOG = 12,         /**< ANIMAL */
    HORSE = 13,       /**< ANIMAL */
    SHEEP = 14,       /**< ANIMAL */
    COW = 15,         /**< ANIMAL */
    CELLPHONE = 16,   /**< ELECTRONICS */
    LAPTOP = 17,      /**< ELECTRONICS */
    BANANA = 18,      /**< FRUIT/VEGETABLE */
    APPLE = 19,       /**< FRUIT/VEGETABLE */
    ORANGE = 20,      /**< FRUIT/VEGETABLE */
    CARROT = 21,      /**< FRUIT/VEGETABLE */
    SPORTSBALL = 23,  /**< SPORT */
    LAST
};

enum class OBJECT_TRACKING_STATE {
    OFF,       /**< The tracking is not yet initialized, the object ID is not usable */
    OK,        /**< The object is tracked */
    SEARCHING, /**< The object couldn't be detected in the image and is potentially occluded, the trajectory is estimated */
    TERMINATE, /**< This is the last searching state of the track, the track will be deleted in the next retreiveObject */
    LAST
};

enum class OBJECT_ACTION_STATE {
    IDLE = 0,   /**< The object is staying static. */
    MOVING = 1, /**< The object is moving. */
    LAST
};
```

### Skeleton body formats

The `skeleton` namespace defines one enumeration of body parts, one `getIdx_*` helper and one list of bones for each of the four body formats supported by the Body Tracking module. The format in use is reported by the `skel_format` field of `ZedObjectData` and is selected with the `bt-format` property of the `zedsrc` element.

| Body format  | Parts enum      | Index helper  | Bones list      |
| ------------ | --------------- | ------------- | --------------- |
| 18 keypoints | `BODY_18_PARTS` | `getIdx_18()` | `BODY_18_BONES` |
| 34 keypoints | `BODY_34_PARTS` | `getIdx_34()` | `BODY_34_BONES` |
| 38 keypoints | `BODY_38_PARTS` | `getIdx_38()` | `BODY_38_BONES` |
| 70 keypoints | `BODY_70_PARTS` | `getIdx_70()` | `BODY_70_BONES` |

The 18-keypoint format, for example, is defined as follows:

**`C++`**

```cpp C++
namespace skeleton {

enum class BODY_18_PARTS {
    NOSE = 0,
    NECK = 1,
    RIGHT_SHOULDER = 2,
    RIGHT_ELBOW = 3,
    RIGHT_WRIST = 4,
    LEFT_SHOULDER = 5,
    LEFT_ELBOW = 6,
    LEFT_WRIST = 7,
    RIGHT_HIP = 8,
    RIGHT_KNEE = 9,
    RIGHT_ANKLE = 10,
    LEFT_HIP = 11,
    LEFT_KNEE = 12,
    LEFT_ANKLE = 13,
    RIGHT_EYE = 14,
    LEFT_EYE = 15,
    RIGHT_EAR = 16,
    LEFT_EAR = 17,
    LAST = 18
};

inline int getIdx_18(BODY_18_PARTS part) { return static_cast<int>(part); }

const std::vector<std::pair<BODY_18_PARTS, BODY_18_PARTS>> BODY_18_BONES{
    {BODY_18_PARTS::NOSE, BODY_18_PARTS::NECK},
    {BODY_18_PARTS::NECK, BODY_18_PARTS::RIGHT_SHOULDER},
    {BODY_18_PARTS::RIGHT_SHOULDER, BODY_18_PARTS::RIGHT_ELBOW},
    {BODY_18_PARTS::RIGHT_ELBOW, BODY_18_PARTS::RIGHT_WRIST},
    {BODY_18_PARTS::NECK, BODY_18_PARTS::LEFT_SHOULDER},
    {BODY_18_PARTS::LEFT_SHOULDER, BODY_18_PARTS::LEFT_ELBOW},
    {BODY_18_PARTS::LEFT_ELBOW, BODY_18_PARTS::LEFT_WRIST},
    {BODY_18_PARTS::RIGHT_SHOULDER, BODY_18_PARTS::RIGHT_HIP},
    {BODY_18_PARTS::RIGHT_HIP, BODY_18_PARTS::RIGHT_KNEE},
    {BODY_18_PARTS::RIGHT_KNEE, BODY_18_PARTS::RIGHT_ANKLE},
    {BODY_18_PARTS::LEFT_SHOULDER, BODY_18_PARTS::LEFT_HIP},
    {BODY_18_PARTS::LEFT_HIP, BODY_18_PARTS::LEFT_KNEE},
    {BODY_18_PARTS::LEFT_KNEE, BODY_18_PARTS::LEFT_ANKLE},
    {BODY_18_PARTS::RIGHT_SHOULDER, BODY_18_PARTS::LEFT_SHOULDER},
    {BODY_18_PARTS::RIGHT_HIP, BODY_18_PARTS::LEFT_HIP},
    {BODY_18_PARTS::NOSE, BODY_18_PARTS::RIGHT_EYE},
    {BODY_18_PARTS::RIGHT_EYE, BODY_18_PARTS::RIGHT_EAR},
    {BODY_18_PARTS::NOSE, BODY_18_PARTS::LEFT_EYE},
    {BODY_18_PARTS::LEFT_EYE, BODY_18_PARTS::LEFT_EAR}
};
}
```

The complete definition of the four formats is available in the [`gstzedmeta.h` header](https://github.com/stereolabs/zed-gstreamer/blob/master/gst-zed-meta/gstzedmeta.h).

> **Note**
>
> An example of how to use Object Detection data is provided with the [`gstzedodoverlay`](/docs/integrations/gstreamer/zed-od-overlay/) [element source code](https://github.com/stereolabs/zed-gstreamer/tree/master/gst-zed-od-overlay).