> For clean Markdown of any page, append .md to the page URL. > For a complete documentation index, see https://docs.stereolabs.com/llms.txt. > For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.stereolabs.com/_mcp/server. # Body Tracking Overview The body tracking module focuses on detecting and tracking a person's bones. A detected bone is represented by its two endpoints, also called keypoints. The ZED camera can provide 2D and 3D information on each detected keypoint. Furthermore, it produces the local rotation between neighboring bones. > **Note** > > This module requires a stereo camera equipped with an inertial sensor (IMU), such as the ZED 2/2i, ZED Mini, ZED X, ZED X Mini, ZED X Nano. The following cameras are **not supported**: > > * The original **ZED** (out of production), which has no built-in IMU. > * The **ZED X One GS/4K/S** cameras, which are monocular. ## How It Works The overall process is very similar to the ZED SDK Object Detection module. They share some output information, such as the 3D position and 3D velocity of each person. The body tracking module also uses a neural network for keypoint detection and then relies on the depth and positional tracking modules of the ZED SDK to compute the final 3D position of each keypoint. The ZED SDK supports multiple body formats: #### Body 18 The `BODY_18` body format contains 18 keypoints following the COCO18 skeleton representation. ![image](/_fern-img/a49e48f73606931fab5f1e507f27ab5d65d644a2791892d6be536094246b1552.webp) Each keypoint is indexed by an integer from 0 to 17: | keypoint index | keypoint name | keypoint index | keypoint name | | :------------: | :-------------: | :------------: | :-----------: | | **0** | NOSE | **9** | RIGHT\_KNEE | | **1** | NECK | **10** | RIGHT\_ANKLE | | **2** | RIGHT\_SHOULDER | **11** | LEFT\_HIP | | **3** | RIGHT\_ELBOW | **12** | LEFT\_KNEE | | **4** | RIGHT\_WRIST | **13** | LEFT\_ANKLE | | **5** | LEFT\_SHOULDER | **14** | RIGHT\_EYE | | **6** | LEFT\_ELBOW | **15** | LEFT\_EYE | | **7** | LEFT\_WRIST | **16** | RIGHT\_EAR | | **8** | RIGHT\_HIP | **17** | LEFT\_EAR | #### Body 34 The `BODY_34` body format contains 34 keypoints following this configuration: ![image](/_fern-img/3a09b701794feeaa249cc64de521686838fe6fdcb43877c56cfafaf8acdc0db9.webp) * Each keypoint is indexed by an integer from 0 to 33: | keypoint index | keypoint name | keypoint index | keypoint name | keypoint index | keypoint name | keypoint index | keypoint name | | :------------: | :------------: | :------------: | :-------------: | :------------: | :-----------: | :------------: | :-----------: | | **0** | PELVIS | **9** | LEFT\_HANDTIP | **18** | LEFT\_HIP | **27** | NOSE | | **1** | NAVAL\_SPINE | **10** | LEFT\_THUMB | **19** | LEFT\_KNEE | **28** | LEFT\_EYE | | **2** | CHEST\_SPINE | **11** | RIGHT\_CLAVICLE | **20** | LEFT\_ANKLE | **29** | LEFT\_EAR | | **3** | NECK | **12** | RIGHT\_SHOULDER | **21** | LEFT\_FOOT | **30** | RIGHT\_EYE | | **4** | LEFT\_CLAVICLE | **13** | RIGHT\_ELBOW | **22** | RIGHT\_HIP | **31** | RIGHT\_EAR | | **5** | LEFT\_SHOULDER | **14** | RIGHT\_WRIST | **23** | RIGHT\_KNEE | **32** | LEFT\_HEEL | | **6** | LEFT\_ELBOW | **15** | RIGHT\_HAND | **24** | RIGHT\_ANKLE | **33** | RIGHT\_HEEL | | **7** | LEFT\_WRIST | **16** | RIGHT\_HANDTIP | **25** | RIGHT\_FOOT | | | | **8** | LEFT\_HAND | **17** | RIGHT\_THUMB | **26** | HEAD | | | #### Body 38 The `BODY_38` body format contains 38 keypoints, including simplified face, hand and foot keypoints, following this configuration: ![image](/_fern-img/61a72b917e708964d7c759d1a6cf41ab5c4349e47f7080037611e9a67bfb047d.webp) * Each keypoint is indexed by an integer from 0 to 37: | keypoint index | keypoint name | keypoint index | keypoint name | keypoint index | keypoint name | keypoint index | keypoint name | | :------------: | :-----------: | :------------: | :-------------: | :------------: | :---------------: | :------------: | :--------------------: | | **0** | PELVIS | **10** | LEFT\_CLAVICLE | **20** | LEFT\_KNEE | **30** | LEFT\_HAND\_THUMB\_4 | | **1** | SPINE\_1 | **11** | RIGHT\_CLAVICLE | **21** | RIGHT\_KNEE | **31** | RIGHT\_HAND\_THUMB\_4 | | **2** | SPINE\_2 | **12** | LEFT\_SHOULDER | **22** | LEFT\_ANKLE | **32** | LEFT\_HAND\_INDEX\_1 | | **3** | SPINE\_3 | **13** | RIGHT\_SHOULDER | **23** | RIGHT\_ANKLE | **33** | RIGHT\_HAND\_INDEX\_1 | | **4** | NECK | **14** | LEFT\_ELBOW | **24** | LEFT\_BIG\_TOE | **34** | LEFT\_HAND\_MIDDLE\_4 | | **5** | NOSE | **15** | RIGHT\_ELBOW | **25** | RIGHT\_BIG\_TOE | **35** | RIGHT\_HAND\_MIDDLE\_4 | | **6** | LEFT\_EYE | **16** | LEFT\_WRIST | **26** | LEFT\_SMALL\_TOE | **36** | LEFT\_HAND\_PINKY\_1 | | **7** | RIGHT\_EYE | **17** | RIGHT\_WRIST | **27** | RIGHT\_SMALL\_TOE | **37** | RIGHT\_HAND\_PINKY\_1 | | **8** | LEFT\_EAR | **18** | LEFT\_HIP | **28** | LEFT\_HEEL | | | | **9** | RIGHT\_EAR | **19** | RIGHT\_HIP | **29** | RIGHT\_HEEL | | | ### About Keypoint Landmarks Keypoint names such as `LEFT_SHOULDER` or `LEFT_KNEE` identify the joint each keypoint represents, but they are not metrologically calibrated anatomical landmarks. Each keypoint's position is the output of a neural network trained to estimate human pose from the camera image and depth data, so it is a **learned approximation of the joint's visible location**, not a precise, fixed point such as the acromion or the glenohumeral joint center. This is a limitation common to all camera-based (RGB or RGB-D) pose estimation systems, since the deep joint center of a real joint (for example, the shoulder's glenohumeral joint) cannot be directly observed from any external camera view. > **Note** > > For applications where absolute joint-center accuracy matters, such as biomechanics or clinical research, treat the ZED SDK skeleton as a consistent **kinematic model** rather than as calibrated anatomical landmarks, and validate keypoint accuracy against ground truth (for example, marker-based motion capture) for your specific setup and population. The ZED SDK can output 3 levels of information: ***raw 2D/3D body detection***, ***3D body tracking*** and ***3D body fitting***. ### 2D/3D Body detection The ZED SDK first uses the ZED camera image to infer all 2D bones and keypoints using neural networks. The SDK depth module and positional tracking module are then used together to extract the correct 3D position of each bone and keypoint. ### 3D body tracking If tracking is enabled, the ZED SDK will assign an identity to each detected body over time. At the same time, by filtering the raw body detections, it will output a more stable 3D body estimation. ### 3D body fitting Moreover, a user can enable fitting to unlock even more information about each identity. The fitting process uses the history of each tracked person to deduce all missing keypoints, relying on the human kinematic constraints used by the body tracking module. It can also extract the local rotation between a pair of neighboring bones by solving the inverse kinematics problem. This data is compatible with common avatar animation software, for example. Below is an example where `BODY_FORMAT::BODY_34` was used to animate an avatar in Unreal. ## Detection Outputs Each detected person is stored as a structure in the ZED SDK called **sl.BodyData**. | Body Data | Description | Output | | :-----------------------------: | :--------------------------------------------------------------------------------------------------: | :-----------------------------------: | | **ID** | Fixed ID for identifying an object over time. | `Integer` | | **Unique Object ID** | Unique id to help identify and track AI detections. | `String` | | **Tracking state** | Defines if an object is currently tracked or lost. | `Ok`, `Off`, `Searching`, `Terminate` | | **Action state** | Defines if an object is currently idle or moving. | `Idle`, `Moving` | | **Position** | Provides the 3D position of the object according to the camera as a 3D vector (x,y,z). | `[x, y, z]` | | **Velocity** | Provides the velocity of the object in space as a 3D vector (x,y,z). | `[vₓ, vᵧ, v_z]` | | **Position covariance** | Covariance matrix of the 3D position. | array of 6 values | | **2D bounding box** | Defines the box surrounding the object in the image represented as four 2D points. | Four pixel coordinates | | **Mask** | Distinguishes the pixels that actually belong to the object from those of the background. | Binary mask | | **Detection confidence** | A lower confidence means the object might not be localized perfectly or that its label is uncertain. | `0 - 100` | | **3D bounding box** | Defines the box surrounding the object in space represented as eight 3D points. | Eight 3D coordinates | | **Dimensions** | Provides the width, height and length of the object. | `[width, height, length]` | | **2D keypoint** | A set of useful points representing the human body, expressed in 2D. | a vector of `[x, y]` | | **Keypoint** | A set of useful points representing the human body, expressed in 3D. | a vector of `[x, y, z]` | | **2D head bounding box** | Bounds the head with four 2D points. | Four pixel coordinates | | **3D head bounding box** | Bounds the head with eight 3D points. | Eight 3D coordinates | | **Head position** | 3D head centroid | `[x, y, z]` | | **Keypoint confidence** | Per keypoint detection confidence | a vector of float | | **Keypoint covariance** | Per keypoint covariance | a vector of 6-value arrays | | **Local position per joint** | Position of each keypoint relative to its parent joint | a vector of `[x, y, z]` | | **Local orientation per joint** | Rotation of each keypoint relative to its parent joint | a vector of `[x, y, z, w]` | | **Global root orientation** | Absolute orientation of the root joint in the camera/world reference frame | `[x, y, z, w]` | See [Understanding Joint Orientations](/docs/development/zed-sdk/modules/body-tracking/using-the-api/#understanding-joint-orientations) for details on how these rotations are defined and how to compose them into world-space orientations. For more information on Body Tracking, see the [Using the API](/docs/development/zed-sdk/modules/body-tracking/using-the-api/) page.