> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.stereolabs.com/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.stereolabs.com/_mcp/server.

# Using the Body Tracking API

Since V4.0, the AI detection module has been split into two different modules: body tracking and object detection. Each module has its own data structures, methods and parameters.
Before that, the body tracking feature was directly integrated into the object detection module.

## Body Tracking Configuration

To configure the body tracking module, use `BodyTrackingParameters` at initialization and `BodyTrackingRuntimeParameters` to change specific parameters during use.

> **Note**
>
> The initial configuration must be set only once when enabling the module, whereas the runtime configuration can be changed at runtime.

* `BodyTrackingParameters::detection_model` selects the human body detection model. This preset configures the runtime and accuracy of the human body detector:

  * `BODY_TRACKING_MODEL::HUMAN_BODY_FAST`: real-time performance even on NVIDIA® Jetson™ or low-end GPU cards

  * `BODY_TRACKING_MODEL::HUMAN_BODY_MEDIUM`: a compromise between accuracy and speed

  * `BODY_TRACKING_MODEL::HUMAN_BODY_ACCURATE`: state-of-the-art accuracy, requires a powerful GPU

* `BodyTrackingParameters::enable_body_fitting`: this enables the fitting process for each detected person. It must be enabled to retrieve the local rotations of each keypoint. Otherwise, the data will be empty.

* `BodyTrackingParameters::body_format` is the body format output by the ZED SDK. The currently supported body formats are:
  * `BODY_FORMAT::BODY_18`: an 18-keypoint body model. This is a COCO18 format and is not directly compatible with public software like Unreal or Unity. For this reason, the local rotation and translation of each keypoint are not available with this format.

  * `BODY_FORMAT::BODY_34`: a 34-keypoint body model. This model is compatible with public software, and all data available for `BODY_18` can also be extracted with this format. The **body\_fitting** option must be enabled to use this format.

  * `BODY_FORMAT::BODY_38`: a 38-keypoint body model. This includes simplified face, hand and foot keypoints.

* `BodyTrackingParameters::model_gen` (`BODY_TRACKING_MODEL_GEN`) selects which neural network generation runs for the chosen `detection_model`:

  * `BODY_TRACKING_MODEL_GEN::GEN_2` *(default)*: a bottom-up network introduced in ZED SDK 5.5, more robust in crowded scenes and to unusual body orientations, and up to 18% faster and 14% more accurate than `GEN_1`. It is only available for `HUMAN_BODY_MEDIUM` and `HUMAN_BODY_ACCURATE`, with `BODY_FORMAT::BODY_18` or `BODY_FORMAT::BODY_34`. It always runs in `FP16`, so `allow_reduced_precision_inference` has no effect on it.

  * `BODY_TRACKING_MODEL_GEN::GEN_1`: the network used up to ZED SDK 5.4. It is the only generation available for `HUMAN_BODY_FAST` and for `BODY_FORMAT::BODY_38`.

  The ZED SDK automatically falls back to the generation it supports when `GEN_2` is requested but not available for the chosen `detection_model`/`body_format` combination.

* `BodyTrackingParameters::body_selection` (`BODY_KEYPOINTS_SELECTION`) selects which keypoints of the chosen body format are output:

  * `BODY_KEYPOINTS_SELECTION::FULL` *(default)*: outputs every keypoint of the body format.

  * `BODY_KEYPOINTS_SELECTION::UPPER_BODY`: outputs only the keypoints from the hips up (arms, head, torso).

* `BodyTrackingParameters::enable_segmentation`: computes a 2D mask distinguishing the pixels of each detected person from the background.

* `BodyTrackingParameters::max_range`: sets an upper depth range (in the `UNIT` configured at `InitParameters` level) beyond which people are not detected. Defaults to `InitParameters::depth_maximum_distance`.

* `BodyTrackingParameters::allow_reduced_precision_inference`: allows the AI model to run at a lower precision (FP16/INT8) to improve runtime and memory usage, at a small (typically 1-2%) cost in accuracy. It has no effect on `GEN_2`, which always runs in `FP16`.

* `BodyTrackingParameters::prediction_timeout_s`: duration, in seconds, during which the ZED SDK keeps predicting a tracked person's position after it stops being detected, before switching its tracking state to `SEARCHING`. Set to `0` to disable prediction.

At runtime, `BodyTrackingRuntimeParameters` also exposes:

* `BodyTrackingRuntimeParameters::detection_confidence_threshold`: minimum detection confidence (1-100) for a body to be output.
* `BodyTrackingRuntimeParameters::minimum_keypoints_threshold`: discards a detected skeleton if fewer than this number of keypoints were detected. Useful for removing unstable fitting results when a person is partially occluded.
* `BodyTrackingRuntimeParameters::skeleton_smoothing`: amount of temporal smoothing applied to the fitted skeleton, from `0` (none) to `1` (maximum smoothing, more latency).

The code below shows how to set the most commonly used attributes.

**`C++`**

```cpp C++
// Set initialization parameters
BodyTrackingParameters detection_parameters;
detection_parameters.detection_model = BODY_TRACKING_MODEL::HUMAN_BODY_ACCURATE; //specific to human skeleton detection
detection_parameters.enable_tracking = true; // Objects will keep the same ID between frames
detection_parameters.enable_body_fitting = true; // Fitting process is called, the user has access to all available data for a person processed by SDK
detection_parameters.body_format = BODY_FORMAT::BODY_34; // selects the 34 keypoints body model for SDK outputs
detection_parameters.model_gen = BODY_TRACKING_MODEL_GEN::GEN_2; // selects the GEN_2 network (default)

// Set runtime parameters
BodyTrackingRuntimeParameters detection_parameters_rt;
detection_parameters_rt.detection_confidence_threshold = 40;
```

**`Python`**

```python Python
# Set initialization parameters
detection_parameters = sl.BodyTrackingParameters()
detection_parameters.detection_model = sl.BODY_TRACKING_MODEL.HUMAN_BODY_ACCURATE  
detection_parameters.enable_tracking = True
detection_parameters.enable_body_fitting = True
detection_parameters.body_format = sl.BODY_FORMAT.BODY_34
detection_parameters.model_gen = sl.BODY_TRACKING_MODEL_GEN.GEN_2 # selects the GEN_2 network (default)

# Set runtime parameters
detection_parameters_rt = sl.BodyTrackingRuntimeParameters()
detection_parameters_rt.detection_confidence_threshold = 40
```

**`C#`**

```csharp C#
// Set initialization parameters
BodyTrackingParameters detection_parameters = new BodyTrackingParameters();
detection_parameters.enableObjectTracking = true; // Objects will keep the same ID between frames
detection_parameters.detectionModel = sl.BODY_TRACKING_MODEL.HUMAN_BODY_ACCURATE;
detection_parameters.enableBodyFitting = true;
detection_parameters.bodyFormat = sl.BODY_FORMAT.BODY_34;
detection_parameters.modelGen = sl.BODY_TRACKING_MODEL_GEN.GEN_2; // selects the GEN_2 network (default)

// Set runtime parameters
BodyTrackingRuntimeParameters detection_parameters_rt = new BodyTrackingRuntimeParameters();
detection_parameters_rt.detectionConfidenceThreshold = 40;
```

If you want to track people's motion within their environment, you will first need to activate the [positional tracking](/docs/development/zed-sdk/modules/positional-tracking/) module. Then, set `detection_parameters.enable_tracking` to `true`.

**`C++`**

```cpp C++
if (detection_parameters.enable_tracking) {
    // Set positional tracking parameters
    PositionalTrackingParameters positional_tracking_parameters;
    // Enable positional tracking
    zed.enablePositionalTracking(positional_tracking_parameters);
}
```

**`Python`**

```python Python
if detection_parameters.enable_tracking:
    # Set positional tracking parameters
    positional_tracking_parameters = sl.PositionalTrackingParameters()
    # Enable positional tracking
    zed.enable_positional_tracking(positional_tracking_parameters)
```

**`C#`**

```csharp C#
if (detection_parameters.enableObjectTracking ) {
    // Set positional tracking parameters
    PositionalTrackingParameters trackingParams = new PositionalTrackingParameters();
    // Enable positional tracking
    zed.EnablePositionalTracking(ref trackingParams);
  }
```

With these parameters configured, you can enable the Body Tracking module:

**`C++`**

```cpp C++
// Enable body tracking with initialization parameters
zed_error = zed.enableBodyTracking(detection_parameters);
if (zed_error != ERROR_CODE::SUCCESS) {
    cout << "enableBodyTracking: " << zed_error << "\nExit program.";
    zed.close();
    exit(-1);
}
```

**`Python`**

```python Python
# Enable body tracking with initialization parameters
zed_error = zed.enable_body_tracking(detection_parameters)
if zed_error != sl.ERROR_CODE.SUCCESS:
    print("enable_body_tracking", zed_error, "\nExit program.")
    zed.close()
    exit(-1)
```

**`C#`**

```csharp C#
// Enable body tracking with initialization parameters
zed_error = zed.EnableBodyTracking(ref detection_parameters);
if (zed_error != ERROR_CODE.SUCCESS) {
    Console.WriteLine("enableBodyTracking: " + zed_error + "\nExit program.");
    zed.Close();
    Environment.Exit(-1);
}
```

> **Note**
>
> The Body Tracking module requires a stereo camera equipped with an inertial sensor (IMU). The original ZED (no IMU) and the monocular ZED X One cameras are not supported. See the [Body Tracking overview](/docs/development/zed-sdk/modules/body-tracking/) for the full list of supported cameras.

## Getting Human Body Data

To get the detected people in a scene, grab a new image with `grab(...)` and extract them with `retrieveBodies()`. This process is exactly the same as getting new objects with the Object Detection module.

**`C++`**

```cpp C++
sl::Bodies bodies; // Structure containing all the detected bodies
// grab runtime parameters
RuntimeParameters runtime_parameters;
runtime_parameters.measure3D_reference_frame = sl::REFERENCE_FRAME::WORLD;

if (zed.grab(runtime_parameters) == ERROR_CODE::SUCCESS) {
  zed.retrieveBodies(bodies, detection_parameters_rt); // Retrieve the detected bodies
}
```

**`Python`**

```python Python
bodies = sl.Bodies() # Structure containing all the detected bodies
# grab runtime parameters
runtime_params = sl.RuntimeParameters()
runtime_params.measure3D_reference_frame = sl.REFERENCE_FRAME.WORLD

if zed.grab(runtime_params) == sl.ERROR_CODE.SUCCESS:
  zed.retrieve_bodies(bodies, detection_parameters_rt) # Retrieve the detected bodies
```

**`C#`**

```csharp C#
sl.Bodies bodies = new sl.Bodies(); // Structure containing all the detected bodies
// grab runtime parameters
RuntimeParameters runtimeParameters = new RuntimeParameters();
runtimeParameters.measure3DReferenceFrame = sl.REFERENCE_FRAME.WORLD;

if (zed.Grab(ref runtimeParameters) == ERROR_CODE.SUCCESS) {
  zed.RetrieveBodies(ref bodies, ref detection_parameters_rt); // Retrieve the detected bodies
}
```

The `sl::Bodies` class stores all data regarding the different people present in the scene in its `vector<sl::BodyData> body_list` attribute. Each person's data is stored as a `sl::BodyData`. `sl::Bodies` also contains the timestamp of the detection, which can help connect the bodies to the images.

All 2D data is related to the left image, while the 3D data is expressed in either the `CAMERA` or `WORLD` reference frame, depending on `RuntimeParameters.measure3D_reference_frame` (given to the `grab()` function). The 2D data is expressed in the initial camera resolution `RESOLUTION`. Scaling can be applied if the value is needed in another resolution.

For 3D data, the coordinate frame and units can be set by the user using `COORDINATE_SYSTEM` and `UNIT`, respectively. These settings are accessible through `InitParameters` when opening the ZED camera with the open function.

### Accessing 2D and 3D body keypoints

Once a `sl::BodyData` is retrieved from the body vector, you can access information such as its ID, position, velocity, label, and tracking\_state but also its keypoint positions and rotations.

The 2D and 3D keypoint data of a detected person are accessible in a vector of pixel keypoints `keypoint_2d` and a vector of 3D positions `keypoint`.

**`C++`**

```cpp C++
// collect all 2D keypoints
for (auto& kp_2d : body.keypoint_2d) {
  // user code using each kp_2d point
}

// collect all 3D keypoints
for (auto& kp_3d : body.keypoint)
{
  // user code using each kp_3d point
}
```

**`Python`**

```python Python
# collect all 2D keypoints
for kp_2d in body.keypoint_2d:
    # user code using each kp_2d point
    pass

# collect all 3D keypoints
for kp_3d in body.keypoint:
    # user code using each kp_3d point
    pass
```

**`C#`**

```csharp C#
// collect all 2D keypoints
foreach (var kp_2d in body.keypoints2D)
{
  // user code using each kp_2d point
}

// collect all 3D keypoints
foreach (var kp_3d in body.keypoints)
{
  // user code using each kp_3d point
}
```

See the keypoint index and name correspondence as well as the output skeleton format [here](/docs/development/zed-sdk/modules/body-tracking/#how-it-works).

#### Getting more results

When fitting is enabled at the initial configuration stage, more results become available according to the chosen `BODY_FORMAT`. The local rotation and translation of each keypoint become available to the user with the `BODY_FORMAT::BODY_34` or `BODY_FORMAT::BODY_38` format.

**`C++`**

```cpp C++
// collect local rotation for each keypoint
for (auto &kp : body.local_orientation_per_joint)
{
   // kp is the local keypoint rotation represented by a quaternion
   // user code
}

// collect local translation for each keypoint
for (auto &kp : body.local_position_per_joint)
{

   // kp is the local keypoint translation
   // user code
}

// get global root orientation
auto global_root_orientation = body.global_root_orientation;

// note that global root translation is available in body.keypoint[root_index] where root_index is the root index of the body model
```

**`Python`**

```python Python
# collect local rotation for each keypoint
for kp in body.local_orientation_per_joint:
    # kp is the local keypoint rotation represented by a quaternion
    # user code
    pass


# collect local translation for each keypoint
for kp in body.local_position_per_joint:
  # kp is the local keypoint translation
  # user code
  pass

# get global root orientation
global_root_orientation = body.global_root_orientation

# note that global root translation is available in body.keypoint[root_index] where root_index is the root index of the body model
```

**`C#`**

```csharp C#
// collect local rotation for each keypoint
foreach (var kp in body.localOrientationPerJoint)
{
   // kp is the local keypoint rotation represented by a quaternion
   // user code
}

// collect local translation for each keypoint
foreach (var kp in body.localPositionPerJoint)
{

   // kp is the local keypoint translation
   // user code
}

// get global root orientation
Quaternion globalRootOrientation = body.globalRootOrientation;
```

> **Note**
>
> Both the keypoint positions and the joint orientations follow the `COORDINATE_SYSTEM` convention configured in `InitParameters`, with positional values expressed in the configured `UNIT`.

#### Understanding Joint Orientations

`local_orientation_per_joint` and `global_root_orientation` are not expressed the same way, and it is important to distinguish them before using them for retargeting or biomechanical analysis:

* **`local_orientation_per_joint`** is the rotation of a keypoint **relative to its parent joint** in the body's kinematic chain (the same parent/child relationship used by `local_position_per_joint`), not relative to the world or the root. For example, the rotation stored for `LEFT_ELBOW` is relative to its parent, `LEFT_SHOULDER`, not relative to the camera or world axes. To reconstruct the **absolute** (world-space) orientation of a given joint, you must compose its local orientation with the local orientations of every ancestor joint up to the root, rather than using a single joint's value on its own.

* The identity quaternion (`[0, 0, 0, 1]`) for a given joint's `local_orientation_per_joint` corresponds to that joint's orientation in the body fitting model's neutral rest pose, which is a **T-pose** (body upright, arms extended horizontally). This is the same reference pose that the [Unreal Engine](/docs/software-integration/unreal-engine-5/body-tracking/) and [Unity](/docs/software-integration/unity/body-tracking/) integrations require a target avatar's skeleton to be rigged in, so that the ZED SDK's local rotations can be applied directly onto the avatar's bones.

* **`global_root_orientation`** is the measured, absolute orientation of the skeleton's root joint (the pelvis for `BODY_34` and `BODY_38`), expressed in the same reference frame (`CAMERA` or `WORLD`, set via `RuntimeParameters.measure3D_reference_frame`) and `COORDINATE_SYSTEM` as the rest of the 3D body data. Because it reflects the person's actual measured orientation, it is generally **not** identity; it is only the identity quaternion at the instant the root joint happens to be aligned with the reference frame's axes.

## Code Example

For code examples, check out the [Tutorial](https://github.com/stereolabs/zed-sdk/tree/master/tutorials) and [Sample](https://github.com/stereolabs/zed-sdk/tree/master/body%20tracking) on GitHub.