> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.stereolabs.com/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.stereolabs.com/_mcp/server.

# Using the Object Detection API

## Object Detection Configuration

To configure the object detection module, use `ObjectDetectionParameters` at initialization and `ObjectDetectionRuntimeParameters` to change specific parameters during use.

**`C++`**

```cpp C++
// Set initialization parameters
ObjectDetectionParameters detection_parameters;
detection_parameters.enable_tracking = true; // Objects will keep the same ID between frames
detection_parameters.enable_segmentation = true; // Outputs 2D masks over detected objects

// Set runtime parameters
ObjectDetectionRuntimeParameters detection_parameters_rt;
detection_parameters_rt.detection_confidence_threshold = 25;
```

**`Python`**

```python Python
# Set initialization parameters
detection_parameters = sl.ObjectDetectionParameters()
detection_parameters.enable_tracking = True # Objects will keep the same ID between frames
detection_parameters.enable_segmentation = True # Outputs 2D masks over detected objects

# Set runtime parameters
detection_parameters_rt = sl.ObjectDetectionRuntimeParameters()
detection_parameters_rt.detection_confidence_threshold = 25
```

**`C#`**

```csharp C#
// Set initialization parameters
ObjectDetectionParameters detection_parameters = new ObjectDetectionParameters();
detection_parameters.enableObjectTracking = true; // Objects will keep the same ID between frames
detection_parameters.enableSegmentation = true; // Outputs 2D masks over detected objects

// Set runtime parameters
ObjectDetectionRuntimeParameters detection_parameters_rt = new ObjectDetectionRuntimeParameters();
detection_parameters_rt.detectionConfidenceThreshold = 25;
```

Various Object ***Box*** detection models are available in the ZED SDK:

* the general purpose object detection including `OBJECT_DETECTION_MODEL::MULTI_CLASS_BOX_FAST`, `OBJECT_DETECTION_MODEL::MULTI_CLASS_BOX_MEDIUM` and `OBJECT_DETECTION_MODEL::MULTI_CLASS_BOX_ACCURATE`. You can choose one of them depending on desired performance/accuracy. These models are able to detect multiple object classes `OBJECT_CLASS`.
* the head detection models `OBJECT_DETECTION_MODEL::PERSON_HEAD_BOX_FAST` and `OBJECT_DETECTION_MODEL::PERSON_HEAD_BOX_ACCURATE`. It is specialized in person head detection and tracking. It may be beneficial for applications in a crowded scene where persons in the background are poorly detected by the general-purpose person detection model. We have separated this model from the general-purpose object detection model and have brought some special optimization and improvements to increase detection and tracking accuracies. It only detects a single class `OBJECT_CLASS::PERSON` with subclass `OBJECT_SUBCLASS::PERSON_HEAD`.

You can use `detection_parameters.detection_model` to set the detection model:

**`C++`**

```cpp C++
// choose a detection model
detection_parameters.detection_model = OBJECT_DETECTION_MODEL::MULTI_CLASS_BOX_FAST;
```

**`Python`**

```python Python
# choose a detection model
detection_parameters.detection_model = sl.OBJECT_DETECTION_MODEL.MULTI_CLASS_BOX_FAST
```

**`C#`**

```csharp C#
// choose a detection model
detection_parameters.detectionModel = sl.OBJECT_DETECTION_MODEL.MULTI_CLASS_BOX_FAST;
```

If you want to track objects' motion within their environment, you will first need to activate the [positional tracking](/docs/development/zed-sdk/modules/positional-tracking/) module. Then, set `detection_parameters.enable_tracking` to `true`.

**`C++`**

```cpp C++
if (detection_parameters.enable_tracking) {
    // Set positional tracking parameters
    PositionalTrackingParameters positional_tracking_parameters;
    // Enable positional tracking
    zed.enablePositionalTracking(positional_tracking_parameters);
}
```

**`Python`**

```python Python
if detection_parameters.enable_tracking :
    # Set positional tracking parameters
    positional_tracking_parameters = sl.PositionalTrackingParameters()
    # Enable positional tracking
    zed.enable_positional_tracking(positional_tracking_parameters)
```

**`C#`**

```csharp C#
if (detection_parameters.enableObjectTracking) {
  // Set positional tracking parameters
  PositionalTrackingParameters trackingParams = new PositionalTrackingParameters();
  // Enable positional tracking
  zed.EnablePositionalTracking(ref trackingParams);
}
```

With these parameters configured, you can enable the object detection module:

**`C++`**

```cpp C++
// Enable object detection with initialization parameters
zed_error = zed.enableObjectDetection(detection_parameters);
if (zed_error != ERROR_CODE::SUCCESS) {
    cout << "enableObjectDetection: " << zed_error << "\nExit program.";
    zed.close();
    exit(-1);
}
```

**`Python`**

```python Python
# Enable object detection with initialization parameters
zed_error = zed.enable_object_detection(detection_parameters)
if zed_error != sl.ERROR_CODE.SUCCESS:
    print("enable_object_detection", zed_error, "\nExit program.")
    zed.close()
    exit(-1)
```

**`C#`**

```csharp C#
// Enable object detection with initialization parameters
zed_error = zed.EnableObjectDetection(ref detection_parameters);
if (zed_error != ERROR_CODE.SUCCESS) {
    Console.WriteLine("enableObjectDetection: " + zed_error + "\nExit program.");
    zed.Close();
    Environment.Exit(-1);
}
```

> **Note**
>
> Object Detection uses the camera motion sensors for improved reliability. Therefore the Object Detection module requires a stereo camera equipped with an IMU (ZED 2/2i, ZED Mini, ZED X, ZED X Mini or ZED X Nano), and inertial sensors cannot be disabled when using the module.

## Getting Object Data

To get the detected objects in a scene, get a new image with `grab(...)` and extract the detected objects with `retrieveObjects()`. The objects' 2D positions are relative to the left image, while the 3D positions are either in the `CAMERA` or `WORLD` reference frame depending on `RuntimeParameters.measure3D_reference_frame` (given to the `grab()` function).

**`C++`**

```cpp C++
sl::Objects objects; // Structure containing all the detected objects
if (zed.grab() == ERROR_CODE::SUCCESS) {
  zed.retrieveObjects(objects, detection_parameters_rt); // Retrieve the detected objects
}
```

**`Python`**

```python Python
objects = sl.Objects() # Structure containing all the detected objects
if zed.grab() == sl.ERROR_CODE.SUCCESS:
  zed.retrieve_objects(objects, detection_parameters_rt) # Retrieve the detected objects
```

**`C#`**

```csharp C#
sl.Objects objects = new sl.Objects(); // Structure containing all the detected objects
RuntimeParameters runtimeParameters = new RuntimeParameters();
if (zed.Grab(ref runtimeParameters) == ERROR_CODE.SUCCESS) {
  zed.RetrieveObjects(ref objects, ref detection_parameters_rt); // Retrieve the detected objects
}
```

The `sl::Objects` class stores all the information regarding the different objects present in the scene in the `object_list` attribute. Each individual object is stored as a `sl::ObjectData` with all information about it, such as bounding box, position, mask, etc. All objects from a given frame are stored in a vector within `sl::Objects`. `sl::Objects` also contains the timestamp of the detection, which can help connect the objects to the images.

You can iterate through the objects as follows:

**`C++`**

```cpp C++
for (auto object : objects.object_list)
  std::cout << object.id << " " << object.position << std::endl;
```

**`Python`**

```python Python
for object in objects.object_list:
  print("{} {}".format(object.id, object.position))
```

**`C#`**

```csharp C#
for (int idx = 0; idx < objects.numObject; idx++)
  Console.WriteLine(objects.objectData[idx].id + " " + objects.objectData[idx].position);
```

Each detected object can be accessed by using its ID as follows:

**`C++`**

```cpp C++
sl::ObjectData object;
objects.getObjectDataFromId(object, 0); // Get the object with ID = 0
```

**`Python`**

```python Python
object = sl.ObjectData()
objects.get_object_data_from_id(object, 0) # Get the object with ID = 0
```

**`C#`**

```csharp C#
sl.ObjectData objectData = new ObjectData();
objects.GetObjectDataFromId(ref objectData, 0); // Get the object with ID = 0
```

### Accessing Object Information

Once an `sl::ObjectData` is retrieved from the object vector, you can access information such as its ID, position, velocity, label, and tracking\_state:

**`C++`**

```cpp C++
unsigned int object_id = object.id; // Get the object id
sl::float3 object_position = object.position; // Get the object position
sl::float3 object_velocity = object.velocity; // Get the object velocity
sl::OBJECT_TRACKING_STATE object_tracking_state = object.tracking_state; // Get the tracking state of the object
if (object_tracking_state == sl::OBJECT_TRACKING_STATE::OK) {
    cout << "Object " << object_id << " is tracked" << endl;
}
```

**`Python`**

```python Python
object_id = object.id # Get the object id
object_position = object.position # Get the object position
object_velocity = object.velocity # Get the object velocity
object_tracking_state = object.tracking_state # Get the tracking state of the object
if object_tracking_state == sl.OBJECT_TRACKING_STATE.OK:
    print("Object {0} is tracked\n".format(object_id))
```

**`C#`**

```csharp C#
int object_id = objectData.id; // Get the object id
Vector3 object_position = objectData.position; // Get the object position
Vector3 object_velocity = objectData.velocity; // Get the object velocity
OBJECT_TRACKING_STATE object_tracking_state = objectData.objectTrackingState; // Get the tracking state of the object
if (object_tracking_state == sl.OBJECT_TRACKING_STATE.OK) {
    Console.WriteLine("Object " + object_id + " is tracked");
}
```

You can also access the confidence of the detection for each object. This confidence depicts the probability of a detected object to really be present in the scene. Therefore, this confidence can be used to post-filter the detected objects. For example, you can ignore objects with a confidence lower than 10 (on a 0-100 scale):

**`C++`**

```cpp C++
for (auto object : objects.object_list) {
  if (object.confidence < 10.f)
    continue;
  // Work with other objects
}
```

**`Python`**

```python Python
for object in objects.object_list:
  if object.confidence < 10:
    continue
  # Work with other objects
```

**`C#`**

```csharp C#
for (int idx = 0; idx < objects.numObject; idx++) {
  if (objects.objectData[idx].confidence < 10f)
    continue;
  // Work with other objects
}
```

### Getting 3D Bounding Boxes

Each detected object contains two bounding boxes: a 2D bounding box and a 3D bounding box. The 2D bounding box is defined in the image frame while the 3D bounding box is provided with the depth information.

The 2D bounding box is represented as four 2D points starting from the top left corner of the object. The 3D bounding box is represented by eight 3D points starting from the top left front corner, as follows:

![image](/_fern-img/c6f75fd9b80e71c180ccf2f75b8eb796b488f3879c9056e6784d609da92da406.webp)

The 2D and 3D bounding boxes are accessible in `sl::ObjectData`:

**`C++`**

```cpp C++
vector<sl::uint2> object_2Dbbox = object.bounding_box_2d; // Get the 2D bounding box of the object
vector<sl::float3> object_3Dbbox = object.bounding_box; // Get the 3D bounding box of the object
```

**`Python`**

```python Python
object_2Dbbox = object.bounding_box_2d; # Get the 2D bounding box of the object
object_3Dbbox = object.bounding_box; # Get the 3D Bounding Box of the object
```

**`C#`**

```csharp C#
Vector2[] object_2Dbbox = objects.objectData[idx].boundingBox2D; // Get the 2D bounding box of the object
Vector3[] object_3Dbbox = objects.objectData[idx].boundingBox; // Get the 3D bounding box of the object
```

### Getting the Object Mask

Each object can also be represented by its mask. The mask includes the pixels within the 2D bounding box that belong to the object. Pixels from the object itself are set to 255 while the pixels of the background are set to 0. You can access the mask of an object with `sl::Mat object_mask = object.mask;`.

![image](/_fern-img/8f39905899840199c46b0628b09333e48c531299ba80eb3ed3401ff35e37903b.webp)

## Code Example

For code examples, check out the [Tutorial](https://github.com/stereolabs/zed-sdk/tree/master/tutorials) and [Sample](https://github.com/stereolabs/zed-sdk/tree/master/object%20detection) on GitHub.