> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.stereolabs.com/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.stereolabs.com/_mcp/server.

# 3D Object Detection Overview

Object detection is the ability to identify objects present in an image. Thanks to depth sensing and 3D information, the ZED camera can provide the 2D and 3D positions of the objects in the scene.

> **Note**
>
> This module requires a stereo camera equipped with an inertial sensor (IMU), such as the ZED 2/2i, ZED Mini, ZED X, ZED X Mini, ZED X Nano. The following cameras are **not supported**:
>
> * The original **ZED** (out of production), which has no built-in IMU.
> * The **ZED X One GS/4K/S** cameras, which are monocular.

Since ZED SDK 3.6, a custom detector can be used with the API. The 2D detection is ingested and the 3D information such as position, 3D bounding box and more are computed. More information on the [Custom Detector page](/docs/development/zed-sdk/modules/object-detection/custom-object-detection).

## How It Works

The ZED SDK uses AI and neural networks to determine which objects are present in the left image. The SDK then computes the 3D position of each object, as well as its bounding box, using data from the depth module. The objects can also be tracked within the environment over time, even if the camera is in motion, thanks to data from the positional tracking module.

### 3D Object Detection

![image](/_fern-img/98b471d5e993b61103c2c2a27f0c6c4573a7b4223b1cb30e1bbf3739944181bf.webp)

The ZED SDK detects all objects present in the images and computes their 3D position and velocity. The distance of the object from the camera is expressed in metric units (ex. meters) and calculated from the back of the left eye of the camera to the scene object.

The SDK also computes a 2D mask that indicates which pixels of the left image belong to the object. From there, the ZED can output the 2D bounding boxes of the objects and accurately compute the 3D bounding boxes with the help of the depth map.

The bounding boxes of the objects can be displayed over the image or the point cloud as depicted in the image above. For more information on how to display the bounding boxes and the object mask, please see our [Object Detection](https://github.com/stereolabs/zed-sdk/tree/master/object%20detection) sample.

### 3D Object Tracking

If the positional tracking module is activated, the ZED SDK can track the objects within the environment. This means that the detected object will keep the same ID throughout the sequence, allowing the objects' paths to be displayed over time. The object detection sample includes a view of these paths, as depicted below:

![image](/_fern-img/d34fb4ddf6706b844554f7de30bab6a5ca238e971f217b794fe742ddbce5cbcd.webp)

## Detection Outputs

Each object is stored as a structure in the ZED SDK. This structure contains all the information regarding a detected object:

| Object Data              | Description                                                                                          | Output                                |
| ------------------------ | ---------------------------------------------------------------------------------------------------- | ------------------------------------- |
| **ID**                   | Fixed ID for identifying an object over time.                                                        | `Integer`                             |
| **Label**                | Identifies the object type.                                                                          | `Person`, `Vehicle`                   |
| **Tracking state**       | Defines if an object is currently tracked or lost.                                                   | `Ok`, `Off`, `Searching`, `Terminate` |
| **Action state**         | Defines if an object is currently idle or moving.                                                    | `Idle`, `Moving`                      |
| **Position**             | Provides the 3D position of the object according to the camera as a 3D vector (x,y,z).               | `[x, y, z]`                           |
| **Velocity**             | Provides the velocity of the object in space as a 3D vector (x,y,z).                                 | `[v_x, v_y, v_z]`                     |
| **Dimensions**           | Provides the width, height and length of the object.                                                 | `[width, height, length]`             |
| **Detection confidence** | A lower confidence means the object might not be localized perfectly or that its label is uncertain. | `0 - 100`                             |
| **2D bounding box**      | Defines the box surrounding the object in the image represented as four 2D points.                   | Four pixel coordinates                |
| **3D bounding box**      | Defines the box surrounding the object in space represented as eight 3D points.                      | Eight 3D coordinates                  |
| **Mask**                 | Provides the pixels which really belong to the object and those of the background.                   | Binary mask                           |

For more information on Object Detection, see the [Using the API](/docs/development/zed-sdk/modules/object-detection/using-the-api/) page.