Skip to navigation

sl::InitParameters Depth Parameters

For more information on these parameters, see the API documentation page.

Depth Mode

Check this page to get more insights about the currently available depth modes for the ZED cameras.

Depth Precision

For the NEURAL depth modes, InitParameters::depth_precision selects the numerical precision of the neural network inference. DEPTH_PRECISION::FP16 is the default. DEPTH_PRECISION::INT8 runs the network at a lower precision, which is faster and uses less GPU memory, at a small accuracy cost.

Since ZED SDK 5.6, all four NEURAL modes have 8-bit models. With DEPTH_PRECISION::INT8:

  • NEURAL_LIGHT and NEURAL_MAX run in FP8 on Jetson Thor, DGX Spark and RTX 40 and RTX 50 series GPUs, which is more accurate than INT8, and in INT8 on Jetson Orin and RTX 30 series GPUs.
  • NEURAL and NEURAL_PLUS run in INT8 on Jetson Orin, Jetson Thor and RTX 30, RTX 40 and RTX 50 series GPUs.
  • On other GPUs, the NEURAL modes run in FP16, NEURAL included.

With NEURAL_MAX, FP16 gives the most detailed depth and INT8 takes up to about half the time. On the GPUs that support FP8, both precisions run in FP8.

The legacy depth modes always run in FP16.

INT8 is always safe to set: the ZED SDK falls back to FP16, and logs it, when the depth mode or the GPU does not support it. Camera::getInitParameters() reports the precision in use, with two exceptions: with NEURAL_MAX, and with NEURAL_LIGHT in INT8 on the GPUs that run it in FP8, it reports the precision requested and the log gives the one in use. It reports DEPTH_PRECISION::LAST when the depth runs at a precision that has no value in DEPTH_PRECISION, such as on the DLA or the PVA (see Depth Compute Device).

InitParameters init_parameters;
init_parameters.depth_mode = DEPTH_MODE::NEURAL;
init_parameters.depth_precision = DEPTH_PRECISION::INT8; // Use INT8 neural depth inference

The INT8 models can be optimized ahead of time, so the first open() does not wait for the optimization: use sl::AI_MODELS::NEURAL_LIGHT_DEPTH_INT8, NEURAL_DEPTH_INT8, NEURAL_PLUS_DEPTH_INT8 or NEURAL_MAX_DEPTH_INT8 with the AI model API, or the -nrlo_light_int8, -nrlo_int8 and -nrlo_plus_int8 options of ZED Diagnostic.

Depth Compute Device

InitParameters::depth_compute_device selects the processor that runs the neural depth network. The default, DEPTH_COMPUTE_DEVICE::AUTO, picks the processor that gives the best depth performance, currently the GPU.

Since ZED SDK 5.6, NEURAL_LIGHT can run on a Jetson accelerator instead of the GPU, leaving the GPU to your application (experimental). Only small steps around the network still run on the GPU.

ValueWhere it runsAvailability
DEPTH_COMPUTE_DEVICE::DLADeep Learning AcceleratorJetson AGX Orin and Orin NX with JetPack 6. NEURAL_LIGHT only: open() fails with ERROR_CODE::INVALID_FUNCTION_PARAMETERS with another depth mode.
DEPTH_COMPUTE_DEVICE::PVAProgrammable Vision AcceleratorJetson AGX Orin and Orin NX with JetPack 6 or 7.2, Jetson Thor with JetPack 7.2. With a depth mode other than NEURAL_LIGHT, the depth runs on the GPU.

The Jetson Orin Nano has neither a DLA nor a PVA.

When the accelerator cannot be used (the device has none, it is not available, or its model is unavailable), the depth runs on the GPU: open() returns ERROR_CODE::CONFIGURATION_FALLBACK, the log says why, and Camera::getInitParameters() reports DEPTH_COMPUTE_DEVICE::GPU. On the accelerator, the network runs at its own precision and depth_precision is ignored.

InitParameters init_parameters;
init_parameters.depth_mode = DEPTH_MODE::NEURAL_LIGHT;
init_parameters.depth_compute_device = DEPTH_COMPUTE_DEVICE::PVA; // Experimental
  • DLA: the DLA model is downloaded on first use, like the other AI models, then prepared for the device, so the first open() takes a few minutes. A DLA has room for a limited number of depth models for the whole system, other applications included. Cameras in the same application can share a model, but cameras in different applications each need their own room, so a Jetson Orin with two DLAs may run as few as two such cameras. A camera that finds no room runs its depth on the GPU.
  • PVA: several cameras and applications can share the PVA; the depth then takes longer and its time can vary from frame to frame. A warning says so when the PVA is already in use as the camera starts. If the PVA fails during the session, the depth runs on the GPU from that frame on: grab() returns ERROR_CODE::CONFIGURATION_FALLBACK once.
  • Containers: both accelerators need --runtime nvidia. The PVA with JetPack 6 also needs --device nvidia.com/pva=all.

Depth Range

Depth range corresponds to the minimum and maximum distance at which the depth of an object can be estimated.

Depth Range by Camera Model

Below are the typical depth ranges and focal lengths for each ZED camera model:

ZED Mini
Focal Length3mm
Max Depth Range0.1m to 15m (0.3ft to 49ft)
Ideal Depth Range0.1m to 9m (0.3ft to 29.5ft)

Minimum Range

You can adjust the minimum detectable depth by setting depth_minimum_distance in InitParameters. Lowering this value allows the camera to estimate depth for objects closer to the lens, within the hardware’s supported range.

With the NEURAL depth modes, the closest depth is also limited by the range of the depth network, which depends on the mode and the camera: for example about 23 cm with NEURAL_MAX, 12 cm with NEURAL_PLUS and 11.5 cm with NEURAL_LIGHT on a ZED X at 1200p. An object closer than that is reported as too close. Since ZED SDK 5.6, NEURAL reaches down to 3 cm on a ZED X Nano.

InitParameters init_parameters;
init_parameters.coordinate_units = UNIT::METER;
init_parameters.depth_minimum_distance = 0.15; // Set the minimum depth perception distance to 15cm

Maximum Range

You can increase the maximum detectable depth by setting depth_maximum_distance in InitParameters. Raising this value allows the camera to estimate depth for objects farther from the lens, up to the hardware’s supported maximum range.

This is useful for applications that require depth information at longer distances.

Depth accuracy decreases with distance, so it’s important to consider the trade-off between range and accuracy. Read more about depth accuracy to understand how it affects your application.

InitParameters init_parameters;
init_parameters.depth_mode = DEPTH_MODE::NEURAL ; // Set the depth mode to NEURAL
init_parameters.coordinate_units = UNIT::METER;
init_parameters.depth_maximum_distance = 40; // Set the maximum depth perception distance to 40m
  • The maximum depth range can be reduced to clamp values above a certain distance. This is useful to reduce depth jitter at long distances.
  • Increasing the maximum range has no impact on memory or FPS.

Depth Stabilization

Depth stabilization is a feature that reduces depth map jitter and enhances accuracy by temporally filtering depth data across multiple frames. It leverages the ZED SDK’s positional tracking to maintain stable depth estimates even when the camera is in motion. The system intelligently distinguishes between static and dynamic regions, ensuring that moving objects are not incorrectly fused, which preserves the integrity of depth information in dynamic scenes.

Depth stabilization is enabled by default. Since it enables positional tracking in the background, you can disable depth stabilization by setting init_parameters.depth_stabilization = 0 to improve computational performance. The value sets the stabilization strength, from 0 (disabled) to 100.

Since ZED SDK 5.6, depth stabilization works better in every depth mode, in two cases: with a moving camera, static surfaces no longer show line artifacts; and moving people and objects are followed sooner and leave fewer ghosts behind them. As a trade-off, thin structures such as foliage can lose a small share of their depth, and people standing almost still can show faint creases.

  • For fixed cameras, we recommend enabling PositionalTrackingParameters::set_as_static while using depth stabilization. This allows the depth stabilizer module to know the camera is static so it can disable visual tracking and reduce the computational load.
  • For applications requiring the Positional Tracking module to be disabled, the depth stabilization parameter must be set to 0; otherwise, the Positional Tracking module is automatically activated in the background.
  • Enabling depth stabilization uses temporal tracking to smooth depth over time. However, if the stabilization strength is set too high and objects or the camera move quickly, the system may fail to keep up, causing “ghosting” or motion trails around fast-moving objects. By default, the Depth Stabilization parameter is set to 30.

sl::RuntimeParameters Depth Parameters

For more information on these parameters, see the API documentation page.

Depth Confidence Filtering

Depth estimation can introduce some uncertainty, resulting in points with varying reliability. While a small number of inaccurate points may not impact most applications, scenarios requiring high precision benefit from filtering out less reliable depth data.

To help with this, the ZED SDK provides a confidence map. Each pixel in the confidence map is assigned a value from 0 (high confidence) to 100 (low confidence). Pixels with higher values are less trustworthy and can be excluded or handled differently to improve the overall quality of your depth data.

sl::Mat confidence_map;
zed.retrieveMeasure(confidence_map, sl::MEASURE::CONFIDENCE);

To filter out unreliable points from the depth map or point cloud, you can either implement your own filtering function using the retrieved confidence data, or simply set the sl::RuntimeParameters::confidence_threshold parameter. When you set this threshold, the ZED SDK automatically removes all points with a confidence value higher than the specified limit, streamlining the process and ensuring only reliable depth data is used.

Two different types of confidence thresholds are available:

  • confidence_threshold: Filters out depth points with low confidence, primarily removing unreliable measurements around object edges. This helps prevent objects that are close together from appearing “linked” in the depth map.
  • texture_confidence_threshold: Filters out depth points in regions with low image texture (uniform or featureless areas), where depth estimation is less reliable due to insufficient visual information.

Since ZED SDK 5.6, the default confidence_threshold also removes many more of the flying pixels of the NEURAL depth modes: the “curtains” of points left floating between an object and the background, most visible behind plants, around people and around screens. A small share of the removed points were valid depth along object edges; to keep them, set confidence_threshold to 98 or above.

ImageOriginal Depth
RGB image of officeDepth map of office
confidence_threshold set to 50texture_confidence_threshold set to 50
Depth with confidence thresholdDepth with texture confidence threshold