> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.stereolabs.com/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.stereolabs.com/_mcp/server.

# ZED SDK 4.0

> **Note**
>
> Release notes of the ZED SDK 4.0 series. Patch releases are listed newest first. Download links and the full changelog, including ZED Tools and integrations, are available on the [ZED SDK 4.0 release page](https://www.stereolabs.com/developers/release/4.0).

## What's New

We are excited to announce the release of ZED SDK 4.0, which introduces a range of new features and enhancements to our ZED cameras. Our latest update supports the ZED X and ZED X Mini cameras, designed specifically for autonomous mobile robots in indoor and outdoor environments. We are also introducing an improved `NEURAL` depth mode, which offers even more accurate depth maps in challenging situations such as low-light environments and textureless surfaces.

We are proud to introduce the new **multi-camera Fusion API**, which makes it easier than ever to fuse data coming from multiple cameras. This module handles time synchronization and geometric calibration issues, along with 360° data fusion with noisy data coming from multiple cameras and sensor sources. We believe that these updates will unlock even more potential for our users to create innovative applications that push the boundaries of what is possible with depth-sensing technology.

For those upgrading from SDK version 3.X to 4.0, we highly recommend checking out our [migration guide](https://www.stereolabs.com/developers/release/4.0/migration-guide) to ensure a smooth transition.

## Major 4.0 API Changes

### Video

* Added new camera models `MODEL::ZED_X` and `MODEL::ZED_XM`.
* Added the new `INPUT_TYPE::GMSL` input type for GMSL cameras (support for GMSL cameras is only available on hosts with Nvidia Jetson SoC).
* Added `INPUT_TYPE input_type` field to `sl::DeviceProperties` struct.
* Added Camera reboot non-blocking behavior. This is enabled using the new `InitParameters::async_grab_camera_recovery` parameter. When the asynchronous automatic camera recovery is enabled and there's an issue with the communication with the camera, the `grab()` will exit after a short period of time, and return the new `ERROR_CODE::CAMERA_REBOOTING` error code. The recovery will run in the background until proper communication is restored. The default and previous ZED SDK behavior is synchronous, the `grab()` function is blocking and will return only once the camera communication is restored or the timeout has been reached.
* `setFromCameraID` and `setFromSerialNumber` methods from `sl::InputType` now take an optional `BUS_TYPE` to choose between USB or GMSL cameras. When unspecified, the method searches for available USB cameras *first*, then searches for GMSL.
* Added new methods `getInputType()`, `getConfiguration()`, and `isInit()` for `sl::InputType`.
* Added a scale utility method to `sl::CameraParameters` to easily convert camera intrinsic parameters for a given resolution.
* Added `RESOLUTION::HD1200` (1920x1200) and `RESOLUTION::SVGA` (960x600) resolutions for ZED X and ZED X Mini cameras.
* Added `RESOLUTION::AUTO` resolution which sets `RESOLUTION::HD720` resolution for USB cameras, and `RESOLUTION::HD1200` resolution for GMSL cameras.
* Added new parameters in `VIDEO_SETTINGS` for GMSL cameras only:
  * `EXPOSURE_TIME`: Image sensor exposure time in ms
  * `ANALOG_GAIN`: Analog sensor gain in dB
  * `DIGITAL_GAIN`: Digital ISP gain in dB
  * `AUTO_EXPOSURE_TIME_RANGE`: Defines the range of the exposure time in automatic control
  * `AUTO_ANALOG_GAIN_RANGE`: Defines the range of sensor gain in automatic control. Min/Max range between \[1000 - 16000] mdB.
  * `AUTO_DIGITAL_GAIN_RANGE`: Defines the range of digital ISP gain in automatic control.
  * `EXPOSURE_COMPENSATION`: Exposure target compensation made after AE. Reduces the overall illumination by a factor of F-stops. values range is \[0 - 100] (mapped between \[-2.0, 2.0]).
  * `DENOISING`: Defines the level of denoising applied on both left and right images. values range is \[0 - 100].
* `Camera::getCameraSettings` now returns an `ERROR_CODE` and uses a reference to retrieve the value.
* `Camera::setCameraSettings` now returns an `ERROR_CODE` instead of void.

### Depth

* `SENSING_MODE` has been removed from the API. To get the same behavior, use the new `sl::RuntimeParameters.enable_fill_mode` parameter.
* Improved Fused Point Cloud extraction time by 2.5x.
* Improved Positional Tracking loop closure time.
* Improved Windows AI model download error handling: when there's no internet access or if the server can't be reached the exact error is now displayed. It will also auto-retry an alternative link if the primary server can't be reached.

### Multi-Camera Fusion API

* Added `sl::Fusion` API in a new header `sl/Fusion.hpp`. In 4.0 Early Access, multi-camera fusion supports multi-camera capture, calibration, and body tracking fusion. All SDK capabilities will be added to the Fusion API in the final 4.0 release.
* The Fusion API introduces *publishers* and *subscribers* to exchange data between source cameras and the fusion module.
* A *publisher* will perform its computations and *publish* its data using the `startPublishing()` method.
* A subscriber will use the `subscribe()` method from the `sl::Fusion` class to connect to all of the publishers and start retrieving data. The `sl::Fusion.process()` method takes care of all the heavy lifting of synchronizing and fusing the data, and produces the enhanced body tracking information in real-time.
* Local workflow: `sl::Fusion` is available for a setup with multiple cameras on a single host, using intra-process communication. The *publishers* and the single *subscriber* run on the same machine in this configuration.
* Distributed workflow: `sl::Fusion` is available for a setup with multiple cameras on multiple hosts, communicating on the local network, enabled by ZED Hub. Each *publisher* and the single subscriber run on separate machines such as ZED Boxes, in order to distribute computations with devices on the edge.

### Geo-tracking API

* The Geo-tracking API is a new module that brings global-scale location tracking to the ZED SDK. It improves the ZED camera's positional tracking capabilities by incorporating additional GNSS data to achieve global positioning. This is done by utilizing the new `sl::Fusion.ingestGNSSdata` method, which integrates the GNSS data with the existing tracking data to create a fused position. The resulting position can be accessed both locally and globally using the getPosition and `getGeoPose` methods, respectively. With this capability, users can achieve highly accurate positioning information that can be used in a variety of applications.
* Added a `GeoPose` structure containing the tracked position in global world reference.
* Added a `GNSSData` structure containing information about GNSS input data.
* Added structures `ECEF`, `LatLng`, and `UTM` for different GPS/GNSS coordinate systems.
* Added `getGeoPose` and `getPosition` methods in `sl::Fusion` that return the fused position in the local and global world frames.
* Added `Geo2Camera` and `Camera2Geo` methods in `sl::Fusion` to convert from a position in the global world reference (world map) frame to a position in the local world frame, and vice-versa.

### Body Tracking

* Body Tracking is now a standalone module, with its own specific parameters and functions.
* The new `BODY_TRACKING_MODEL` enum contains the following models:
  * `HUMAN_BODY_FAST`: Keypoints based, specific to human skeleton, real-time performance even on Jetson or low-end GPU cards.
  * `HUMAN_BODY_MEDIUM`: Keypoints based, specific to human skeletons, compromise between accuracy and speed.
  * `HUMAN_BODY_ACCURATE`: Keypoints based, specific to human skeletons, state-of-the-art accuracy, requires powerful GPU.
* The model now depends both on the accuracy wanted and on the `BODY_FORMAT` (see below).
* Changed `BODY_FORMAT` namings from `POSE_XX` to `BODY_XX`. The `POSE_18` and `POSE_34` formats have been renamed to `BODY_18` and `BODY_34` respectively.
* Compared to ZED SDK 3.8, the `HUMAN_BODY_FAST` and `HUMAN_BODY_ACCURATE` models when using `BODY_18` and `BODY_34` are slower but more precise. Up to 50% more precise for the fast model with a runtime difference of around 2 ms on Jetson Orin.
* Added a new human body format in `BODY_FORMAT`:
  * `BODY_38`: Adds additional key points to `BODY_34`, specifically on the feet and hands. The body fitting now provides accurate orientations for the feet and hands.
* `BODY_18` and `BODY_34`: The previous body models are still available
* Added the possibility to use multiple body tracking models at once using the instance_id parameter in `sl::BodyTrackingParameters` (and in related methods such as `Camera::getBodyTrackingParameters`, `Camera::disableBodyTracking`, ...).
* Added a `BODY_KEYPOINTS_SELECTION` enum to filter key points for specific use cases. Currently available options include `BODY_KEYPOINTS_SELECTION::FULL` and `BODY_KEYPOINTS_SELECTION::UPPER_BODY`.
* Analogous to object detection, body-tracking objects now have their own methods and parameters exposed separately:
  * Added a new `sl::BodyData` class which contains detection and tracking information on detected human bodies, such as keypoint, joint, and skeleton root information.
  * Added a new `Camera::retrieveBodies` method to retrieve human body detection data after a `Camera::grab` call, similar to `Camera::retrieveObjects`.
  * Added a new `sl::Bodies` class which is returned from `Camera::retrieveBodies`, similar to `sl::Objects`.
  * Added a new `sl::BodyTrackingParameters` and `sl::BodyTrackingRuntimeParameters` classes which define all of the parameters related to body tracking.
* Renamed `BODY_PARTS` to `BODY_18_PARTS`.
* Renamed `BODY_PARTS_POSE_34` to `BODY_34_PARTS`.
* Added `BODY_38_PARTS` which lists all key points for the new body format.
* Renamed `BODY_BONES` to `BODY_18_BONES`.
* Renamed `BODY_BONES_POSE_34` to `BODY_34_BONES`.
* Added `BODY_38_BONES` bones containers for new body format.

### Object Detection

* Object detection models have been separated into *object detection models* and *human body detection models*, in two separate classes: `OBJECT_DETECTION_MODEL` and `BODY_TRACKING_MODEL` respectively.
* The new `OBJECT_DETECTION_MODEL` enum contains the following models:
  * `MULTI_CLASS_BOX_FAST` : Any objects, bounding box based.
  * `MULTI_CLASS_BOX_MEDIUM` : Any objects, bounding box based, a compromise between accuracy and speed.
  * `MULTI_CLASS_BOX_ACCURATE` : Any objects, bounding box based, more accurate but slower than the base model.
  * `PERSON_HEAD_BOX_FAST` : Bounding Box detector specialized in the person's head, particularly well suited for crowded environments, the person localization is also improved.
  * `PERSON_HEAD_BOX_ACCURATE` : Bounding Box detector specialized in the person's head, particularly well suited for crowded environments, the person localization is also improved, more accurate but slower than the base model.
  * `CUSTOM_BOX_OBJECTS` : For external inference, by using your own custom model and/or frameworks. This mode disables the internal inference engine, the 2D bounding box detection must be provided.
* Added the possibility to use multiple object detection models instances at once using the `instance_id` parameter in `sl::ObjectDetectionParameters` (and in related methods such as `Camera::getObjectDetectionParameters`, `Camera::disableObjectDetection` , ...).
* Renamed `MULTI_CLASS_BOX` to `MULTI_CLASS_BOX_FAST` in `OBJECT_DETECTION_MODEL` .
* Renamed `PERSON_HEAD_BOX` to `PERSON_HEAD_BOX_FAST` in `OBJECT_DETECTION_MODEL` .
* Renamed `enable_mask_output` to `enable_segmentation` in `sl::ObjectDetectionParameters` .
* Removed `keypoint_2d`, `keypoint`, `keypoint_confidence`, `local_position_per_joint`, `local_orientation_per_joint`, and `global_root_orientation` from `sl::ObjectData` . These attributes have been moved to the `sl::BodyData` class for Body Tracking.

### Known Issues

* Maxwell and Kepler GPUs (compute capabilities 35, 50, and 52) are not supported using CUDA 11.x, only with CUDA 10.2

## 4.0.8

### SDK

* Improved Visual-Inertial Positional Tracking for ZED X.
* The Magnetometer heading rotation now compensates for the rotation.
* Resolved ZED X service access issues in Docker.
* Improved GeoTracking initialization process.

### Bug Fixes

* Improved the reliability of concurrent camera openings on Linux.
* Fixed `Camera2Geo` returning a wrong geoposition when position is not in `sl::COORDINATE_SYSTEM::IMAGE`.
* Fixed a random memory leak in GeoTracking during initialization.
* Improved the reliability of the `sl::Camera::reboot` function.
* Fixed `sl::Camera::reboot` function, which was consistently returning `CAMERA_NOT_DETECTED` on Windows.
* Fixed a tracking issue with custom object detection that resulted in poor results with multiple small objects.
* Fixed an issue with `sl::Camera::resetPositionalTracking` to keep the previous state of the parameter `set_gravity_as_origin`.
* Improved the camera opening time that could increase a lot when using multiple cameras on Windows.

## 4.0.7

### SDK

* Added an automatic SVO repair mechanism when opening a corrupted SVO file. The repair is done in place if possible to avoid duplication.
* Added automatic ROI exclusion computation of static objects. It requires motion to determine which part of the image is moving with the camera. This is typically used when the ZED is on a moving vehicle with a visible part in the image. A new `sl::Camera::startRegionOfInterestAutoDetection` function is used to initiate the detection. With enough motion and frames grabbed, it will be automatically applied by default and also available with `sl::Camera::getRegionOfInterest`. `sl::RegionOfInterestParameters` have been added to fine-tune the detection.
* Improved the newly released Positional Tracking Mode `QUALITY` to make it work faster and with all depth modes.
* Improved depth confidence to more accurately estimate low-quality depth pixels, such as flying pixels.
* Improved USB reliability on Linux by increasing the camera V4L2 priority. This should help avoid dropping frames, especially in case of heavy CPU loads.
* Improved the runtime of the Spatial Mapping module, especially on Jetson. On Orin AGX, the process from data integration to mesh retrieval can be up to 10 times faster compared to previous versions.
* Timestamps and dates are now logged in UTC format.
* Added `sl::Camera::isCameraSettingSupported` method for camera video control support.
* Added a new optional corrupted frame detector. This experimental feature can be enabled with `sl::InitParameters::enable_image_validity_check`. It is performed in the `sl::Camera::grab()` method and requires some computations. If an issue is found, the method will output a warning as the new `sl::ERROR_CODE::CORRUPTED_FRAME`. It's currently able to catch USB corrupted frames such as green or purple-tinted frames or buffer corruption. It could also interpret widely different left and right images as invalid.

### Fusion

* Added a `sl::Fusion::stopPublishing` method to the Fusion API.
* Reduced camera grab time when publishing data to Fusion.
* Added synchronized images and depth retrieval from within the Fusion API.

### Bug Fixes

* Fixed incorrect GNSS calibration when the main camera is not at the origin.
* Fixed the fusion blocking when the publisher closes and reopens.
* Fixed a crash in the SDK logger in verbose mode.
* Fixed Positional Tracking that could miss movements in some situations.
* Fixed `sl::Fusion::getPosition` returning incorrect output when multiple cameras are set up.

### Support

* Added L4T 35.4 compatibility.

## 4.0.6

### SDK

* Improved GNSS-VIO Calibration and fusion.
  * Added several functions to make it easier to use: `sl::Fusion::getCurrentGNSSCalibrationSTD` / `sl::Fusion::getGeoTrackingCalibration`.
  * Added new options to control its behavior: `target_translation_uncertainty` / `target_yaw_uncertainty` / `enable_rolling_calibration`. Added new `GNSS_CALIBRATION_STATE` to get more details about its current state.
  * Added timestamp into GeoPose.
* Added function `sl::Fusion::getCurrentTimeStamp` to retrieve the timestamp of the current processed data.
* Added Spatial Mapping module into Fusion API. You can now map your environment with multiple ZED cameras at the same time.
* Improved Spatial Mapping accuracy, better reconstruction of shapes, improved detail when getting closer.
* Added new attribute `sl::CameraParameters::focal_length_metric` that stores the real focal length in millimeters.
* Added `sl::PlaneDetectionParameters` to better control the plane at hit function.
* Improved GMSL image capture pipeline, requires fewer resources and improves the latency.
* Improved GMSL SVGA resolution stability.
* Added support for specific carrier board (that are not using muxer before deserializer).
* Switched USB to max priority on Linux (previously "2" = `INTERACTIVE`, now "3" = `RECORD`).
* Improved USB stability on Linux.
* Added GMSL autorecovery.

### Bug Fixes

* Fixed random issue when opening simultaneously multiple cameras.
* Fixed random camera motion sensor not detected during camera opening.
* Fixed timeout issue and automatic recovery when using 2 ZED X simultaneously.
* Fixed SVO decoding random issue on H26X lossless SVO on Jetsons.
* Fixed SVO recording missing IMU data for ZED X.
* Fixed missing parameters in `sl::Camera::getInitParameters` in SVO playback mode.

## 4.0.5

### SDK

* Introduced low ZED X camera framerate, adding 15 FPS for all available resolutions. Requires Jetson driver update (v0.5.4)
* Added a new `sl::InitParameters::grab_compute_capping_fps` parameter to set an upper computation frequency limit to the grab function. This can be useful to get a known constant fixed rate or limit the computation load while keeping a short exposure time by setting a high camera capture framerate for instance.

### Bug Fixes

* Resolved a bug that caused the GMSL IMU orientation to become invalid (not a number or not a valid quaternion) in some cases. This could also lead to SVO file reading issue since the frames were considered corrupted and skipped.

## 4.0.4

### SDK

* Introduced a new `sl::POSITIONAL_TRACKING_MODE`, set with `sl::PositionalTrackingParameters::mode`. By default the `STANDARD` mode is enabled and identical to the previous implementation. The new `QUALITY` mode enables higher accuracy, especially in challenging cases such as low-feature environments or repetitive textures. Please note that it requires more computation and is for now only available with `ULTRA` depth mode.
* Improved Custom Object Detection module with better error logging and smarter diagnostic. Added security to avoid getting stuck when no depth is computed, invalid box input, invalid multi-instance id, and added an internal timeout when no data are available.
* Added color output to Spatial Mapping Mesh
* Added stability counter parameter in Spatial Mapping module. This defines the number of times each point has to be seen to be integrated into the spatial model. Decreasing the value will increase reactivity and integration speed at the cost of accuracy and noise increase.
* Improved `sl::Camera::setRegionOfInterest` function, it now accepts multiple channel images and provides a warning when the format is invalid instead of undefined behavior. Please note that it’s internally converted to grayscale images and only values equal to 0 are ignored.
* Added new parameter `sl::BodyTrackingRuntimeParameters::skeleton_smoothing` to configure the smoothing of the fitted skeleton, which ranges from 0 (low smoothing) to 1 (high smoothing).

### Bug Fixes

* Fixed an issue that could occur with `NEURAL` depth mode where black vertical lines would corrupt the depth map.
* Fixed regression in object detection module with `MULTI_CLASS_BOX*` models on Jetson L4T 32.7, introduced in 4.0.3.

## 4.0.3

### SDK

* Added GMSL support for NVIDIA Jetson L4T 35.3 (JetPack 5.1.1).
* Fixed new ZED Mini rectification that could lead to an image artifact.
* Improved body fitting for `sl::BODY_FORMAT::BODY_38`, it is now smoother and prevents the model from getting stuck in an unusual position.
* Reduced body jittering when standing still.
* Improved Object Detection accuracy.
* Improved AI overall loading time.

## 4.0.2

### SDK

* Added new `sl::FUSION_ERROR_CODE` to provide more detailed feedback while using the Geo-tracking module.
* Updated default streaming bitrate to keep a constant quality regardless of the selected resolution/FPS/encoder.

### Bug Fixes

* Fixed Geo-tracking module initialization.
* Fixed an issue leading to inconsistent hip positions with the body tracking Fusion module.
* Fixed body tracking `ACTION_STATE` parameter when body tracking is not activated.

### Support

* Added CUDA 12.1 support for Windows.

## 4.0.1

### Bug Fixes

* Fixed body tracking fusion with `sl::BODY_FORMAT::BODY_34`.
* Fixed inconsistent behavior of `sl::BodyTrackingFusionRuntimeParameters::enable_body_fitting`.
* Fixed `sl::BodyTrackingFusionRuntimeParameters::skeleton_smoothing` to be applied to `sl::BODY_FORMAT::BODY_38`.
* `sl::Camera::getCameraSettings` and `sl::Camera::setCameraSettings` now return `sl::ERROR_CODE` for non available settings (particularly with ZED X cameras).
* Fixed logger file creation without write permission.

### Support

* Jetson running on JetPack 5.1 (L4T 35.2) now supports ZED X cameras.
* Added ZED SDK support for JetPack 5.1.1 (L4T 35.3), ZED X support coming soon.
* Added CUDA 12.0 partial support, CUDA 12.1 general support coming soon.

## 4.0.0 (Early Access)

### ZED X Drivers

Added support for the new ZED X and ZED X Mini cameras.

Introduced new `VIDEO_SETTINGS`, `RESOLUTION`, and `sl::InputType` parameters to enable users to use the same code for both GMSL or USB 3.0 cameras without any added complexity.

### Multi-Camera Fusion

Introduced new Multi-Camera Fusion API. The new module allows seamless synchronization and integration of data from multiple cameras in real time, providing more accurate and reliable information than a single camera setup. Additionally, the Fusion module offers redundancy in case of camera failure or occlusions, making it a reliable solution for critical applications.

The new API can be found in the header `sl/Fusion.hpp` as `sl::Fusion` API. In 4.0 Early Access, multi-camera fusion supports multi-camera capture, calibration, and body tracking fusion. All SDK capabilities will be added to the Fusion API in the final 4.0 release.