Skip to navigation

ROS 2 - ZED Stereo Node

Open in ClaudeOpen in ChatGPT

The easiest way to start a ZED ROS 2 node for a stereo camera is by using the command line:

ros2 launch zed_wrapper zed_camera.launch.py camera_model:=<camera model>

where <camera model> must be replaced with one in the list zed, zedm, zed2, zed2i, zedx, zedxm, zedxnano, and virtual.

Published Topics

Not all the topics are advertised by default. The version v5.1.0 of the ROS 2 Wrapper introduces new parameters to enable/disable the publishing of each topic. Please refer to the “Configuration parameters” section for more details.

The ZED node publishes data to the following topics:

  • Status

    • ~/status/health: the general status of the node (custom message).
    • ~/status/heartbeat: heartbeat signal to monitor if the node is alive from other nodes (custom message).
    • ~/status/svo: the status of the SVO playback (custom message). Only advertised when an SVO file is used as input.
  • Image streams

    The image topic names are created using the following naming convention: ~/<sensor_type>/<color_model>/<rect_type>/image

Starting from version 5.1.0 of the ROS 2 Wrapper the new convention replaces the previous formats such as ~/<sensor_type>/image_<rect_type>_<color_model> and ~/<sensor_type>_<rect_type>/image_<rect_type>_<color_model> (e.g., ~/left/image_rect_color, ~/rgb_raw/image_raw_gray). Please update your launch files and scripts to use the new topic names.

  • RGB channel

    • ~/rgb/color/rect/image: Color rectified image.
    • ~/rgb/color/rect/camera_info: Color camera calibration data.
    • ~/rgb/gray/rect/image: Grayscale rectified image.
    • ~/rgb/gray/rect/camera_info: Grayscale camera calibration data.
    • ~/rgb/color/raw/image: Color unrectified image.
    • ~/rgb/color/raw/camera_info: Color raw calibration data.
    • ~/rgb/gray/raw/image: Grayscale unrectified image.
    • ~/rgb/gray/raw/camera_info: Grayscale raw calibration data.
  • Left channel

    • ~/left/color/rect/image: Left sensor RGB rectified image.
    • ~/left/color/rect/camera_info: Left sensor RGB camera calibration data.
    • ~/left/gray/rect/image: Left sensor Grayscale rectified image.
    • ~/left/gray/rect/camera_info: Left sensor Grayscale camera calibration data.
    • ~/left/color/raw/image: Left sensor RGB unrectified image.
    • ~/left/color/raw/camera_info: Left sensor RGB raw calibration data.
    • ~/left/gray/raw/image: Left sensor Grayscale unrectified image.
    • ~/left/gray/raw/camera_info: Left sensor Grayscale raw calibration data.
  • Right channel

    • ~/right/color/rect/image: Right sensor RGB rectified image.
    • ~/right/color/rect/camera_info: Right sensor RGB camera calibration data.
    • ~/right/gray/rect/image: Right sensor Grayscale rectified image.
    • ~/right/gray/rect/camera_info: Right sensor Grayscale camera calibration data.
    • ~/right/color/raw/image: Right sensor RGB unrectified image.
    • ~/right/color/raw/camera_info: Right sensor RGB raw calibration data.
    • ~/right/gray/raw/image: Right sensor Grayscale unrectified image.
    • ~/right/gray/raw/camera_info: Right sensor Grayscale raw calibration data.

Since v5.5.0 of the ROS 2 Wrapper the distortion_model and the distortion coefficients published in the camera_info topics are read from the ZED SDK (>= 5.4) instead of a hard-coded per-model table that the lens does not always follow: a fisheye ZED X One now reports equidistant instead of rational_polynomial. Rectified parameters are reported as plumb_bob with zero coefficients.

The rgb and left channels of stereo cameras are identical. We use both names to differentiate the usage: while left and right are used for stereo processing, rgb is associated with the synchronized depth map.

  • Stereo pair

    • ~/stereo/color/rect/image: side-by-side left/right rectified stereo pair. Calibration data are available in the topics: ~/left/color/rect/camera_info and ~/right/color/rect/camera_info.
    • ~/stereo/color/raw/image: side-by-side left/right unrectified stereo pair. Calibration data are available in the topics: ~/left/color/raw/camera_info and ~/right/color/raw/camera_info.
  • Depth and point cloud

    • ~/depth/camera_info: Depth camera calibration data.
    • ~/depth/depth_registered: Depth map image registered on left image.
    • ~/image/depth_image: colorized depth image, for visualization only (published only when depth.publish_depth_image is true). Since v5.6.0 of the ROS 2 Wrapper.
    • ~/depth/depth_info: minimum and maximum depth information (custom message).
    • ~/point_cloud/cloud_registered: Registered color point cloud.
    • ~/confidence/confidence_map: Confidence image (floating point values).
    • ~/disparity/map: disparity map, of type stereo_msgs/msg/DisparityImage, to be used for depth processing.
    • ~/disparity/image: normalized disparity map, of type sensor_msgs/msg/Image, for visualization purposes. This is the topic to subscribe to in RViz 2.

The topic ~/disparity/disparity_image is obsolete since v5.4.0 of the ROS 2 Wrapper and is replaced by the two topics above. To visualize the disparity map, run ros2 run image_view disparity_view --ros-args -r image:=/zed/zed_node/disparity/map.

  • Region of Interest

    • ~/roi_mask/image: the Region of Interest mask applied to the ZED SDK processing. See the Region of Interest page.
  • Sensors data

    • ~/left_cam_imu_transform: Transform from the left camera sensor to the IMU sensor.
    • ~/imu/data: Accelerometer, gyroscope, and orientation data in Earth frame.
    • ~/imu/data_raw: Accelerometer and gyroscope data in Earth frame.
    • ~/imu/mag: Raw magnetometer data (only ZED 2 and ZED 2i).
    • ~/atm_press: atmospheric pressure (only ZED 2 and ZED 2i).
    • ~/temperature/imu: temperature measured by the IMU sensor.
    • ~/temperature/left: temperature measured near the left CMOS sensor (only ZED 2 and ZED 2i).
    • ~/temperature/right: temperature measured near the right CMOS sensor (only ZED 2 and ZED 2i).
  • Positional tracking

    • ~/pose: Absolute 3D position and orientation relative to the Map frame (Sensor Fusion algorithm + SLAM).
    • ~/pose/status: the status of the positional tracking module (custom message).
    • ~/pose/motion_prior_links: the links held by the motion prior between keyframes, as a visualization_msgs/Marker for RViz 2 (published only when pos_tracking.motion_prior.enabled is true) [ZED SDK >= 5.6]. See Motion prior.
    • ~/pose_with_covariance: Camera pose referred to Map frame with covariance.
    • ~/odom: Absolute 3D position and orientation relative to the Odometry frame (visual-inertial odometry on cameras equipped with an IMU, pure visual odometry on the original ZED).
    • ~/odom/status: the status of the odometry from the positional tracking module (custom message).
    • ~/path_map: Sequence of camera poses in Map frame.
    • ~/path_odom: Sequence of camera odometry poses in Map frame.
    • ~/pose/landmarks: the 3D landmarks used by the positional tracking algorithm, published as a sensor_msgs/msg/PointCloud2 message.
  • Geo Tracking

    • ~/pose/filtered: the GNSS fused pose of the robot
    • ~/pose/filtered/status: the status of the GNSS fused pose (custom message).
    • ~/geo_pose: the Latitude/Longitude pose of the robot
    • ~/geo_pose/status: the status of the Latitude/Longitude pose (custom message).
    • ~/pose/fused_fix: the GNSS fused fix of the robot, published as a sensor_msgs/msg/NavSatFix message.
    • ~/pose/origin_fix: the Latitude/Longitude of the origin of the map frame, published as a sensor_msgs/msg/NavSatFix message.
  • 3D Mapping

    • ~/mapping/fused_cloud: Fused point cloud created when the enable_mapping service is called.
  • Object Detection

  • Body Tracking

  • Plane Detection

    • ~/plane: Detected plane (custom message).
    • ~/plane_marker: Detected plane object to be displayed on RViz 2.

Each topic has a common prefix created by the launch file as ~ = /<namespace>/<node_name>/, where <namespace> is replaced with the value of the parameter camera_name and <node_name> is the name of the node, e.g. zed_node.

Image Transport

The ROS 2 wrapper supports the stack image_transport introduced with ROS 2 Crystal Clemmys.

Starting from version 5.3 of the ROS 2 Wrapper, image messages are published through rclcpp::TypeAdapter for more efficient handling:

  • The base raw topic is handled by a TypeAdapter publisher. Intra-process subscribers (e.g., other nodes running in the same component container) receive a StampedSlMat object (a wrapper around the native sl::Mat) directly, without any serialization overhead. Inter-process subscribers transparently receive a standard sensor_msgs/msg/Image message thanks to the automatic TypeAdapter conversion.
  • The transport-specific variants are handled by an image_transport publisher. Its raw transport is disabled to avoid publishing duplicate messages, since the raw stream is already provided by the TypeAdapter publisher described above.

To prevent silent data corruption caused by incompatible plugin/encoding combinations (e.g., applying JPEG compression to floating-point depth data), the advertised image transports are filtered by topic type:

  • Visual image topics (color and grayscale images) advertise only:
    • compressed: JPEG or PNG compressed image
    • theora: Ogg Theora compressed image
    • zstd: lossless Zstandard-compressed image (available starting with ROS 2 Jazzy Jalisco)
  • Measurement topics (e.g., depth) advertise only:
    • compressedDepth: depth image compressed with a lossless algorithm
    • zstd: lossless Zstandard-compressed image (available starting with ROS 2 Jazzy Jalisco)

For proper association between images and their corresponding camera information, the ZED ROS 2 Wrapper also publishes camera_info topics for all compressed image streams:

  • ~/rgb/color/rect/image/camera_info
  • ~/rgb/gray/rect/image/camera_info
  • ~/rgb/color/raw/image/camera_info
  • ~/rgb/gray/raw/image/camera_info

When the isaac_ros_nitros package is installed and NITROS is not disabled via the debug.disable_nitros parameter, NITROS publishers take priority and neither the TypeAdapter nor the image_transport publishers are created. Please refer to the dedicated section about NVIDIA® Isaac ROS Nitros for more details.

NVIDIA® Isaac ROS Nitros

The ZED ROS 2 Wrapper supports NITROS with NVIDIA® Isaac™ ROS 3.2. Isaac™ ROS 5 removed the NITROS packages: with Isaac™ ROS 5, the ZED node publishes standard ROS 2 messages and IPC is enabled automatically. See the ZED Isaac™ ROS Integration Overview.

The ZED ROS 2 Wrapper supports the NVIDIA® Isaac™ ROS Nitros package for zero-copy transport of ROS messages using NVIDIA® GPUDirect® RDMA.

NVIDIA® Isaac ROS Nitros integration is enabled and used only if the required packages are manually installed. They are not added as a dependency because this is an optional feature.

Starting from version 5.3 of the ROS 2 Wrapper, Intra-Process Communication (IPC) is handled automatically:

  • If NITROS is not installed on the host machine, IPC is always enabled.
  • If NITROS is installed, IPC is disabled and NITROS takes priority.

If you want to use IPC on a machine where NITROS is installed, set the debug.disable_nitros parameter to true to disable NITROS; IPC will then be enabled automatically. (This parameter has no effect when NITROS is not installed, since IPC is already active.)

IPC is also controlled by the enable_ipc launch argument of zed_camera.launch.py (default true). When NITROS is installed and debug.disable_nitros is false, IPC and NITROS conflict: with the default ipc_nitros_conflict_policy:=disable_ipc the launch file logs a warning and forces enable_ipc:=false for that launch. To use NITROS, you can also set enable_ipc:=false explicitly, as required by the Isaac ROS 3 integration. Set enable_ipc:=false as well when no component shares the container with the ZED node and you do not need intra-process delivery.

For more information about the installation and usage of the NVIDIA® Isaac™ ROS packages, please refer to the dedicated section of the documentation: ZED Isaac™ ROS Integration Overview.

Point Cloud Transport

The ROS 2 wrapper supports the stack pointcloud_transport introduced with ROS 2 Humble Hawksbill.

The point cloud transport is enabled and used only if the required packages are manually installed, they are not added as a dependency because this is an optional feature:

sudo apt install ros-${ROS_DISTRO}-point-cloud-transport \
ros-${ROS_DISTRO}-point-cloud-transport-plugins \
ros-${ROS_DISTRO}-zlib-point-cloud-transport \
ros-${ROS_DISTRO}-zstd-point-cloud-transport \
ros-${ROS_DISTRO}-draco-point-cloud-transport

Point cloud topics are published using the point_cloud_transport::Publisher object, which provides both standard raw and compressed point cloud streams for efficient data transmission.

CUDA zero-copy transport

Starting with v5.6.0, on ROS 2 Lyrical and newer, the ZED ROS 2 Wrapper supports the rosidl::Buffer message fields and shares the data retrieved on the GPU without copies. In ROS 2 Lyrical, the uint8[] data field of messages such as sensor_msgs/msg/Image and sensor_msgs/msg/PointCloud2 is a rosidl::Buffer, whose memory can live on the GPU: the cuda buffer backend shares this GPU memory between a publisher and its subscribers.

The general.buffer_backend parameter selects how the data are published:

  • cuda (default): the images, the depth map, the confidence map, the colorized depth image, and the point cloud stay in GPU memory. The subscribers that accept the cuda backend and run on the same host, with the same GPU and the same Linux user, receive the GPU memory of the ZED node without copies. All the other subscribers, such as RViz2, ros2 bag, Python nodes, and nodes on other hosts, automatically receive standard CPU data, with no change on their side.
  • cpu: the node always publishes standard CPU data, as in the previous versions.

The disparity map is always published from CPU memory. An invalid value is replaced by cuda, with a warning. On ROS 2 Humble and Jazzy, the parameter is ignored.

The CUDA zero-copy transport requires the cuda_buffer package when the wrapper is built, the cuda_buffer_backend plugin when the node runs, and the rmw_fastrtps_cpp middleware (Fast DDS), the default of ROS 2 Lyrical, or the rmw_zenoh_cpp middleware (Zenoh) v0.10.5 or newer. When a requirement is missing, the node publishes standard CPU data. At startup, the node logs the transport in use in the line that starts with * Buffer transport:, and the reason when it falls back to CPU data.

With the cuda backend, the node still publishes standard CPU data, for all the subscribers of a topic, in these cases:

  • An rclcpp Intra-Process Communication (IPC) subscriber is connected to the topic: IPC delivers the messages without the buffer backend negotiation. To receive CUDA buffers in a node loaded in the ZED container, disable IPC for that node.
  • The topic uses the TRANSIENT_LOCAL durability, not supported by the cuda backend.
  • A compressed point_cloud_transport plugin is subscribed to the point cloud: the uncompressed point cloud is then published with CPU data too.

The compressed image_transport topics always carry CPU data, and do not change how the uncompressed image topics are published. The topics published with NITROS are not affected.

The CUDA zero-copy transport guide of the Isaac™ ROS 5 integration explains how the transport works and how to set it up. To receive the CUDA buffers in your own C++ nodes, follow the image and the point cloud tutorials.

QoS profiles

All the topics are configured by default to use the following ROS 2 QoS profile:

  • Reliability: RELIABLE
  • History (Depth): KEEP_LAST (10)
  • Durability: VOLATILE
  • Lifespan: Infinite
  • Deadline: Infinite
  • Liveliness: AUTOMATIC
  • Liveliness lease duration: Infinite

Read the official ROS 2 documentation about the QoS and the related settings for details.

The QoS settings can be modified by changing the relative parameters in the YAML files, as described in this official ROS 2 design document.

When creating a subscriber, be sure to use a compatible QoS profile according to the following tables:

Compatibility of QoS durability profiles:

PublisherSubscriberConnectionResult
VolatileVolatileYesVolatile
VolatileTransient localNo-
Transient localVolatileYesVolatile
Transient localTransient localYesTransient local

Compatibility of QoS reliability profiles:

PublisherSubscriberConnectionResult
Best effortBest effortYesBest effort
Best effortReliableNo-
ReliableBest effortYesBest effort
ReliableReliableYesReliable

In order for a connection to be made, all of the policies that affect compatibility must be compatible. For instance, if a publisher-subscriber pair has compatible reliability QoS profiles, but incompatible durability QoS profiles, the connection will not be made.

For a detailed list of all the compatibility settings, please refer to the official ROS 2 documentation.

Configuration parameters

Specify your launch parameters in the common_stereo.yaml, zed.yaml, zedm.yaml, zed2.yaml, zed2i.yaml, zedx.yaml, zedxm.yaml, zedxnano.yaml, and virtual.yaml files available in the folder zed_wrapper/config.

General parameters

Namespace: general

common_stereo.yaml

ParameterDescriptionValue
camera_timeout_secSeconds between two attempts to open the camera at start-upint
camera_max_reconnectAttempts to open the camera at start-up. Since v5.6.0 of the ROS 2 Wrapper, it also sets how long the node waits for a camera disconnected while running: camera_timeout_sec x camera_max_reconnect secondsint
camera_flipFlip the camera data if it is mounted upside downtrue, false
exit_on_failureSince v5.6.0 of the ROS 2 Wrapper. true stops the whole process when the node fails for good, false stops this node only. Set by zed_camera.launch.py, see When the node stopstrue, false
self_calibEnable the self-calibration process at camera opening. See the API documentationtrue, false
serial_numberSelect a ZED camera by its Serial Numberint
camera_idSelect a ZED camera by its ID (0 for first, 1 for second…)int
virtual_serial_numbersSerial numbers of the two ZED X One cameras composing a Virtual Stereo Camera, [left_sn,right_sn]. Set by the serial_numbers launch argumentint array
virtual_camera_idsIDs of the two ZED X One cameras composing a Virtual Stereo Camera, [left_id,right_id]. Set by the camera_ids launch argumentint array
pub_resolutionThe resolution used for image and depth map publishing. ‘NATIVE’ to use the same general.grab_resolution - CUSTOM to apply the general.pub_downscale_factor downscale factor to reduce bandwidth in transmissionstring,‘NATIVE’, ‘CUSTOM’
pub_downscale_factorRescale factor used to rescale image before publishing when ‘pub_resolution’ is ‘CUSTOM’double
grab_compute_capping_fpsUpper limit applied to the grab frequency to cap the computation load. Set to 0 to disable (process data at the grab rate). With ZED SDK >= 5.6 the newest image is processed at each capped grab (older versions could deliver it up to one capping period late), and the capping also applies to stream inputsdouble
pub_frame_rate*Set the video/depth publish ratedouble
sync_pub_to_timestampSince v5.6.0 of the ROS 2 Wrapper. Select the frames to publish on their image timestamp, so that several GMSL cameras capturing at the same instant publish the same captures. See Synchronized publishing of GMSL camerastrue, false
enable_image_validity_checkSets the image validity check. If set to 1, the SDK will check if the frames are valid before processingint
gpu_idSelect a GPU device for depth computationint
optional_opencv_calibration_fileOptional path where the ZED SDK can find a file containing the calibration information of the camera computed by OpenCV.string
async_image_retrievalIf set to true, the camera images are retrieved at a framerate different from the grab() application framerate. This is useful for recording an SVO or sending the camera stream at a different rate than the applicationtrue, false
buffer_backend[ROS 2 Lyrical and newer] rosidl::Buffer backend used to publish the images, depth maps, confidence maps, and point clouds. cuda keeps the data in GPU memory and shares it without copies with the subscribers on the same host and GPU that accept the cuda backend, while all the other subscribers automatically receive CPU data; cpu always publishes CPU data. Ignored on older ROS 2 distributions. See CUDA zero-copy transportstring, ‘cuda’, ‘cpu’
publish_statusEnable publishing of the node status topicstrue, false
sdk_timestamp_clock[ZED SDK >= 5.3] Clock source behind every SDK timestamp (images and sensors). SYSTEM_CLOCK is the wall clock and is affected by NTP/PTP, MONOTONIC_CLOCK is immune to clock steps but still frequency-slewed by NTP/PTP, MONOTONIC_RAW_CLOCK [ZED SDK >= 5.4] is immune to both steps and slewing. Process-wide: the first node in a composition winsstring, ‘SYSTEM_CLOCK’, ‘MONOTONIC_CLOCK’, ‘MONOTONIC_RAW_CLOCK’
max_system_clock_step_ms[ZED SDK >= 5.3] Largest backward host clock adjustment (NTP/PTP step, manual date change) that the SDK applies to a single sample, in milliseconds. It therefore sets how fast the SDK catches up with a clock step: 4.0 is the ZED SDK default and suits slew-only setups, a negative value makes step corrections land instantly, as PTP deployments expect, and 0.0 freezes the offset captured on first use. It is applied only when sdk_timestamp_clock is SYSTEM_CLOCK, and it is always overridden by the ZED_SDK_MAX_SYSTEM_CLOCK_STEP_MS environment variabledouble
timestamp_reference[ZED SDK >= 5.5] Time reference used to stamp the published data. IMAGE is the start of the sensor readout; IMAGE_CENTER_OF_EXPOSURE is the middle of the frame exposure, half an exposure earlier, which aligns the frames with sensors that are timestamped at the instant they measure, such as a LiDAR or the joints of a robotstring, ‘IMAGE’, ‘IMAGE_CENTER_OF_EXPOSURE’
sdk_use_monotonic_clock[DEPRECATED] Use sdk_timestamp_clock instead. Still read for backward compatibility: when true and sdk_timestamp_clock is left at SYSTEM_CLOCK, the SDK clock is set to MONOTONIC_CLOCK. The node logs a deprecation warningtrue, false

* Dynamic parameter

timestamp_reference: 'IMAGE_CENTER_OF_EXPOSURE' requires an input that carries a per-frame exposure, that is a ZED X or a ZED X Mini, live, from an SVO or from a stream. On the USB models the node warns once and falls back to IMAGE. It is not applied to the sensors data, to the SVO /clock topic, nor in simulation.

Since v5.6.0 of the ROS 2 Wrapper, when the camera is disconnected while the node is running, the node waits general.camera_timeout_sec x general.camera_max_reconnect seconds (25 by default) for it to come back. If it does, the node resumes publishing; otherwise, the node closes the camera and stops as described in When the node stops. Previous versions either waited forever while the ZED SDK was reopening the camera, or were killed as soon as the ZED SDK reported the camera as not detected.

The same time is given to the ZED SDK to reopen a USB camera (InitParameters::open_timeout_sec), so a camera plugged back at any time during the wait resumes; as a consequence, each attempt to open the camera at start-up can also last up to that time.

While the camera is disconnected, the node diagnostic reports Camera disconnected: waiting for it to come back (N / 25 sec), and the Disconnections entry counts the disconnections of the session with the time of the last reconnection.

zed.yaml, zedm.yaml, zed2.yaml, zed2i.yaml, zedx.yaml, zedxm.yaml, zedxnano.yaml, and virtual.yaml

ParameterDescriptionValue
camera_modelType of StereoLabs camerastring, zed, zedm, zed2, zed2i, zedx, zedxm, zedxnano, virtual
camera_nameUser name for the camera, can be different from camera modelstring
grab_resolutionThe native camera grab resolution. The valid values depend on the camera model, see the table below. An invalid value is rejected with a warning and replaced by ‘AUTO’string
grab_frame_rateZED SDK internal grabbing rateint, ‘15’,‘30’,‘60’,‘100’,‘120’

Valid grab_resolution values per camera model:

Camera modelValid values
zed, zedm, zed2, zed2i’HD2K’, ‘HD1080’, ‘HD720’, ‘VGA’, ‘AUTO’
zedx, zedxm, zedxnano, virtual’HD1200’, ‘HD1080’, ‘SVGA’, ‘AUTO’

When the node stops

Since v5.6.0 of the ROS 2 Wrapper, a node that fails for good (the camera cannot be opened at start-up or does not come back after a disconnection or a reboot, invalid configuration, critical camera error) closes the camera and stops cleanly. The general.exit_on_failure parameter decides what happens next:

  • true: the whole process stops. zed_camera.launch.py sets this value when it creates the container, so the launch then ends and ros2 launch returns. The process exits with code 0, so a supervisor must restart it on any exit (e.g. systemd Restart=always, not Restart=on-failure).
  • false (default): only this node stops, and the other nodes of the same container keep running, for example the other cameras started with zed_multi_camera.launch.py. zed_camera.launch.py sets this value when the node is loaded in an existing container (container_name launch argument). The node diagnostic reports Node stopped with the reason; unload the node with ros2 component unload before loading it again.

Previous versions stopped the whole container when a node could not open its camera, usually with a crash, which also stopped the cameras of the other nodes in it.

SVO input parameters

Namespace: svo

common_stereo.yaml

ParameterDescriptionValue
svo_pathSpecify SVO filenamestring, relative file paths are not allowed
svo_encoding_preset[ZED SDK >= 5.3] SVO encoder speed/quality preset used by the start_svo_rec servicestring, ‘DEFAULT’, ‘ULTRAFAST’, ‘FAST’, ‘MEDIUM’, ‘SLOW’
encryption_key[ZED SDK >= 5.3] Optional key/passphrase used to encrypt the SVO files created by the start_svo_rec service (AES-256-CTR). Accepted as a path to a 32-byte key file, a 64-character hex key, or a passphrase. Empty disables encryptionstring
decryption_key[ZED SDK >= 5.3] Optional decryption key/passphrase used when opening encrypted SVO filesstring
use_svo_timestampsUse the SVO timestamps to publish data. If false, data will be published at the system timetrue, false
publish_svo_clockWhen use_svo_timestamps is true allows you to publish the SVO clock to the /clock topic. This is useful for synchronous rosbag playbacktrue, false
svo_loopRestart the SVO if the end of the file is reachedtrue, false
svo_realtimeif true SVO will be played trying to respect the original framerate eventually skipping frames, otherwise every frame will be processed respecting the pub_frame_rate settingtrue, false
play_from_frameStart playing the SVO from a specific frame IDint
replay_rate*Replay rate for the SVO when not used in realtime modedouble, [0.10-5.0]

* Dynamic parameter

SVO encryption requires OpenSSL (libcrypto) at runtime. The value of svo.encryption_key is never written to the log, and the same value must be set as svo.decryption_key to play the recording back: if the key is lost the recorded data is unrecoverable.

Streaming input parameters

Namespace: stream

common_stereo.yaml

ParameterDescriptionValue
stream_addressAddress of the local streaming serverstring
stream_portPort of the local streaming server (only even values are valid)int

Simulation input parameters

Namespace: simulation

common_stereo.yaml

ParameterDescriptionValue
sim_enabledEnable simulation inputtrue, false
sim_addressAddress of the simulation serverstring
sim_portPort of the simulation serverint
self_calibSince v5.6.0 of the ROS 2 Wrapper. Enable the self-calibration at camera opening with a simulated camera, in place of general.self_calib. Disabled by default, for reproducible simulation resultstrue, false

Since v5.6.0 of the ROS 2 Wrapper [ZED SDK >= 5.6 on the simulator and on the node side], the video parameters (video.*, including their dynamic changes) are sent to the simulated camera, and the simulator applies the ones it supports. A setting the simulator does not support is reported once at the INFO level and not applied. With older ZED SDK versions the video parameters are ignored in simulation. See Using the ZED with ROS 2 in Isaac™ Sim to set up the simulated camera.

Streaming server parameters

Namespace: stream_server

common_stereo.yaml

ParameterDescriptionValue
stream_enabledenable the streaming server when the camera is opentrue, false
codecDifferent encoding types for image streaming: ‘H264’, ‘H265’string
portPort used for streaming. Port must be an even number. Any odd number will be rejected.int
bitrateStreaming bitrate (in Kbits/s) used for streaming. See docint, [1000 - 60000]
gop_sizeThe GOP size determines the maximum distance between IDR/I-frames. Very high GOP size will result in slightly more efficient compression, especially on static scenes. But latency will increase.int, [max 256]
adaptative_bitrateBitrate will be adjusted depending the number of packet dropped during streaming. If activated, the bitrate can vary between [bitrate/4, bitrate].true, false
chunk_sizeStream buffers are divided into X number of chunks where each chunk is chunk_size bytes long. You can lower chunk_size value if network generates a lot of packet lost: this will generates more chunk for a single image, but each chunk sent will be lighter to avoid inside-chunk corruption. Increasing this value can decrease latency.int, [1024 - 65000]
target_framerateFramerate for the streaming output. This framerate must be below or equal to the camera framerate. Allowed framerates are 15, 30, 60 or 100 if possible. Any other values will be discarded and camera FPS will be taken.int

Video parameters

Namespace: video

common_stereo.yaml

ParameterDescriptionValue
saturation*Image saturationint, [0,8]
sharpness*Image sharpnessint, [0,8]
gamma*Image gammaint, [0,8]
auto_exposure_gain*Enable the auto exposure and auto gaintrue, false
exposure*Exposure value if auto-exposure is falseint [0,100]
gain*Gain value if auto-exposure is falseint [0,100]
auto_whitebalance*Enable the auto white balancetrue, false
whitebalance_temperature*White balance temperatureint [28,65]
enable_24bit_outputEnable BGR 24-bit output for images (instead of 32-bit)true, false
publish_rgbEnable publishing of RGB image streamstrue, false
publish_left_rightEnable publishing of Left and Right image streamstrue, false
publish_rawEnable publishing of raw (unrectified) image streamstrue, false
publish_grayEnable publishing of grayscale image streamstrue, false
publish_stereoEnable publishing of stereo pair image streamstrue, false

* Dynamic parameter

zed.yaml, zed2.yaml, and zed2i.yaml

ParameterDescriptionValue
brightness*Image brightnessint, [0,8]
contrast*Image contrastint, [0,8]
hue*Image hueint, [0,11]

* Dynamic parameter

zedx.yaml, zedxm.yaml, zedxnano.yaml, and virtual.yaml

ParameterDescriptionValue
exposure_time*Defines the real exposure time in microseconds. Recommended to control manual exposure (instead of video.exposure setting)int, [28,66000]
auto_exposure_time_range_min*Defines the minimum range of exposure auto control in micro secondsint, [28,66000]
auto_exposure_time_range_max*Defines the maximum range of exposure auto control in micro secondsint, [28,66000]
exposure_compensation*Defines the Exposure-target compensation made after auto exposure. Reduces the overall illumination target by a factor of F-stops. 50 means no compensation. Since v5.6.0, -1 leaves the camera value unchanged (the node does not set it)int, [-1 - 100]
analog_gain*Defines the real analog gain (sensor) in mDB. Recommended to control manual sensor gain (instead of video.gain setting)int, [1000-16000].
auto_analog_gain_range_min*Defines the minimum range of sensor gain in automatic controlint
auto_analog_gain_range_max*Defines the maximum range of sensor gain in automatic controlint
digital_gain*Defines the real digital gain (ISP) as a factor. Recommended to control manual ISP gain (instead of video.gain setting)int, [1-256]
auto_digital_gain_range_min*Defines the minimum range of digital ISP gain in automatic controlint
auto_digital_gain_range_max*Defines the maximum range of digital ISP gain in automatic controlint
denoising*Defines the level of denoising applied on both left and right imagesint, [0-100]
ae_antibanding*[ZED SDK >= 5.3] Auto-exposure anti-banding mode, used to remove the flicker produced by artificial lightingint, -1=not set (since v5.6.0, the camera keeps its own value), 0=OFF, 1=AUTO, 2=50Hz, 3=60Hz

* Dynamic parameter

The node accepts exposure_time and the auto_exposure_time_range_* values from 28 to 66,000 µs, but the camera applies only the exposure range of the video mode: from 39 µs (21 µs in SVGA) to one frame period, up to 65,000 µs at 15 fps. With ZED SDK 5.6, a value outside this range is not applied, and the node logs no warning.

Region of Interest parameters

Namespace: region_of_interest

common_stereo.yaml

ParameterDescriptionValue
automatic_roiEnable the automatic ROI generation to automatically detect part of the robot in the FoV and remove them from the processing. Note: if enabled the value of manual_polygon is ignoredtrue, false
depth_far_threshold_metersFiltering how far object in the ROI should be considered, this is useful for a vehicle for instancedouble
image_height_ratio_cutoffBy default consider only the lower half of the image, can be useful to filter out the skydouble
manual_polygonA polygon defining the ROI where the ZED SDK perform the processing ignoring the rest. Coordinates must be normalized to ‘1.0’ to be resolution independent.string
apply_to_depthApply ROI to depth processingtrue, false
apply_to_positional_trackingApply ROI to positional tracking processingtrue, false
apply_to_object_detectionApply ROI to object detection processingtrue, false
apply_to_body_trackingApply ROI to body tracking processingtrue, false
apply_to_spatial_mappingApply ROI to spatial mapping processingtrue, false
publish_roi_maskPublish the ROI mask imagetrue, false

Depth parameters

Namespace: depth

common_stereo.yaml

ParameterDescriptionValue
depth_modeSelect depth map quality. If ‘NONE’, all the modules that require depth extraction are disabled (Object Detection, Body Tracking, Mapping, …), and so is the Positional Tracking unless pos_tracking.pos_tracking_mode is GEN_3 [ZED SDK >= 5.2]. The ZED SDK CUSTOM mode, where the application provides the depth, is not supported: the node warns and uses NEURALstring, ‘NONE’, ‘NEURAL_LIGHT’, ‘NEURAL’, ‘NEURAL_PLUS’, ‘NEURAL_MAX’ [ZED SDK >= 5.6]
depth_precision[ZED SDK >= 5.5] Precision of the neural depth inference. INT8 is faster and uses less GPU memory than FP16, for a small accuracy cost. With ZED SDK >= 5.6, it is available with every NEURAL depth mode on NVIDIA® Jetson™ Orin, Jetson Thor, DGX Spark and RTX 30, 40 and 50 series GPUs; with ZED SDK 5.5, with NEURAL only. Always safe to set: the ZED SDK falls back to FP16 when the depth mode or the GPU does not support INT8string, ‘FP16’, ‘INT8’
depth_compute_device[ZED SDK >= 5.6] Experimental. Processor that runs the neural depth network. AUTO (currently the GPU) and GPU run it on the GPU; DLA (Jetson AGX Orin and Orin NX with JetPack 6) and PVA (Jetson AGX Orin and Orin NX with JetPack 6 or 7.2, Jetson Thor with JetPack 7.2) run NEURAL_LIGHT on an accelerator of the Jetson at its own precision, which leaves the GPU to the application (depth_precision is ignored). With another depth mode the node warns and uses AUTO; when the accelerator cannot be used, at start-up or during the session, the depth runs on the GPU, the node warns and the diagnostic shows GPUstring, ‘AUTO’, ‘GPU’, ‘DLA’, ‘PVA’
allow_depth_cuda_graph[ZED SDK >= 5.5] Let the ZED SDK record the depth computation once and replay it at every grab, lowering the CPU cost of launching it. This matters most on NVIDIA® Jetson™. The depth output is unchanged. It applies to the NEURAL depth mode family only, and the SDK reverts to the regular computation if the recording failstrue, false
depth_stabilizationEnable depth stabilization. Stabilizing the depth requires an additional computation load as it enables tracking. A negative value takes the SDK default, 0 disables itint, [0,100]
openni_depth_modeEnable/disable the depth OpenNI format (16 bit, millimeters)true, false
point_cloud_freq*frequency of the pointcloud publishing (equal or less to grab_frame_rate value)double
point_cloud_resThe resolution used for point cloud publishing - ‘COMPACT’-Standard resolution. Optimizes processing and bandwidth, ‘REDUCED’-Half resolution. Low processing and bandwidth requirementsstring, ‘COMPACT’, ‘REDUCED’
depth_confidence*Depth confidence threshold (lower values = more filtering)int [0,100]
depth_texture_conf*Depth texture confidence threshold (lower values = more filtering)int [0,100]
remove_saturated_areas*If true color saturated areas are not included in the depth maptrue, false
publish_depth_mapEnable publishing of depth map streamtrue, false
publish_depth_imageSince v5.6.0 of the ROS 2 Wrapper. Enable publishing of the colorized depth image (sl::VIEW::DEPTH) on ~/image/depth_image, for visualization only. It follows video.enable_24bit_output like the other image topics. Disabled by defaulttrue, false
publish_depth_infoEnable publishing of depth info streamtrue, false
publish_point_cloudEnable publishing of point cloud streamtrue, false
voxel_point_cloud*[ZED SDK >= 5.3] When true, publish a voxel-decimated point cloud instead of the full resolution onetrue, false
voxel_size_mm*[ZED SDK >= 5.3] Voxel grid cell size in millimeters. -1 uses the 100 mm default. Clamped to [5, max depth range]double
voxel_resolution_mode[ZED SDK >= 5.3] Voxelization modestring, ‘FIXED’, ‘STEREO_UNCERTAINTY’, ‘LINEAR’
voxel_resolution_scale*[ZED SDK >= 5.3] Scale factor for depth-adaptive voxel growthdouble, [0.01, 3.0]
publish_depth_confidenceEnable publishing of depth confidence map streamtrue, false
publish_disparityEnable publishing of disparity image streamtrue, false

To disable depth calculation set depth_mode to NONE. All the other parameters relative to depth will be ignored and only the video streams and sensors data will be published.

* Dynamic parameter

zed.yaml, zedm.yaml, zed2.yaml, zed2i.yaml, zedx.yaml, zedxm.yaml, zedxnano.yaml, and virtual.yaml

ParameterDescriptionValue
min_depthMinimum depth value (from the camera) that will be computeddouble
max_depthMaximum range allowed for depth. Values beyond this limit will be reported as TOO_FARdouble, the maximum allowed value depends on the camera model (see the comment in its YAML file)

* Dynamic parameter

Positional tracking parameters (Odometry and Localization)

Namespace: pos_tracking

common_stereo.yaml

ParameterDescriptionValue
pos_tracking_enabledEnable positional tracking when the node startstrue, false
pos_tracking_modeAUTO uses the SDK default. Use GEN_1 for the legacy behavior, GEN_3 for the latest algorithmsstring, ‘AUTO’, ‘GEN_1’, ‘GEN_3’
compute_preference[ZED SDK >= 5.5] How much GPU the positional tracking may use. PREFER_GPU makes the tracking faster and shortens grab(), PREFER_CPU keeps the GPU free for your own workloads. GEN_3 runs on the CPU by default; GEN_1 computes depth and so uses the GPU whatever the settingstring, ‘AUTO’, ‘PREFER_CPU’, ‘PREFER_GPU’
imu_fusionEnable IMU fusion. When set to false, only the optical odometry will be used.true, false
publish_tfEnable publishing of transform odom -> <camera_name>_camera_link and all the other transformstrue, false
publish_map_tfEnable publishing of transform map -> odom. Only valid if publish_tf is truetrue, false
map_frameFrame_id of the pose messagestring
odometry_frameFrame_id of the odom messagestring
area_memoryEnable Loop Closing algorithmtrue, false
area_file_pathPath to the area memory file used for relocalization and loop closure in a previously explored environment. Since v5.6.0 of the ROS 2 Wrapper, it is ignored with a warning when area_memory is false: the file is neither loaded nor saved. With GEN_3, files saved with ZED SDK >= 5.6 cannot be loaded by older versions, see Area memory filesstring
enable_localization_onlyIf true, the camera only localizes in the loaded area memory without updating the map with new informationtrue, false
save_area_memory_on_closingSave the area memory before closing the camera if area_file_path is not empty (can also be triggered anytime with the save_area_memory service)true, false
reset_odom_with_loop_closureEnable the automatic reset of the odometry when a loop closure event is detected.true, false
publish_3d_landmarksEnable publishing of the 3D landmarks used by the positional tracking algorithmtrue, false
publish_lm_skip_framePublish the landmarks every X frames to reduce bandwidth. Set to 0 to publish all landmarksint
depth_min_rangeSet this value for removing fixed zones of the robot in the FoV of the camera from the visual odometry evaluationdouble
set_as_staticIf ‘true’ the camera will be static and not move in the environmenttrue, false
set_gravity_as_originIf ‘true’ align the positional tracking world to imu gravity measurement. Keep the yaw from the user initial pose.true, false
floor_alignmentIndicates if the floor must be used as origin for height measurestrue, false
initial_base_poseInitial reference pose. Useful if the base_link does not start from the originarray
path_pub_rate*Frequency (Hz) of publishing of the path messagesdouble
path_max_countMaximum number of poses kept in the pose arrays (-1 for infinite)int
two_d_modeEnable if robot moves on a planar surface. Forces Z to fixed_z_value, roll and pitch to zerotrue, false
fixed_z_valueFixed value assigned to Z coordinate if two_d_mode is truedouble
transform_time_offset*The value added to the timestamp of map->odom and odom->base_link transform being generateddouble
reset_pose_with_svo_loopReset the camera pose the initial_base_pose when the SVO loop is enabled and the SVO playback reaches the end of the filetrue, false
motion_prior.enabled[ZED SDK >= 5.6] Subscribe to motion_prior.topic and send the motion of your own source to the GEN_3 positional tracking. Not available with an SVO file or a simulated camera. See Motion priortrue, false
motion_prior.topic[ZED SDK >= 5.6] nav_msgs/Odometry topic of the motion sourcestring
motion_prior.measured_axes[ZED SDK >= 5.6] Axes measured by the source. Never list an axis the source does not measurestring, e.g. ”[‘TX’,‘TY’,‘RZ’]“
motion_prior.rejection_threshold[ZED SDK >= 5.6] How far the motion may disagree with the images before it is discarded, as a multiple of its uncertaintydouble
motion_prior.translation_std_per_m[ZED SDK >= 5.6] Translation uncertainty per meter, used when the twist.covariance of the messages is not setdouble
motion_prior.rotation_std_per_rad[ZED SDK >= 5.6] Rotation uncertainty per radian, used when the twist.covariance of the messages is not setdouble
publish_odom_poseEnable publishing of odometry and pose messagestrue, false
publish_pose_covEnable publishing of pose_with_covariance messagetrue, false
publish_cam_pathEnable publishing of path_odom messagetrue, false

* Dynamic parameter

Global Localization (GNSS Fusion)

Namespace: gnss_fusion

common_stereo.yaml

ParameterDescriptionValue
gnss_fusion_enabledEnable the fusion of ‘sensor_msgs/NavSatFix’ message information into pose datatrue, false
gnss_fix_topicName of the GNSS topic of type NavSatFix to subscribe [Default: “/fix”]string
gnss_zero_altitudeSet to true to ignore GNSS altitude informationtrue, false
h_covariance_mulMultiplier factor to be applied to horizontal covariance of the received fix (plane X/Y)double
v_covariance_mulMultiplier factor to be applied to vertical covariance of the received fix (Z axis)double
publish_utm_tfPublish utm -> map TFtrue, false
broadcast_utm_transform_as_parent_frameIf ‘true’ publish utm -> map TF, otherwise map -> utmtrue, false
enable_reinitializationDetermines whether reinitialization should be performed between GNSS and VIO fusion when a significant disparity is detected between GNSS data and the current fusion data. It becomes particularly crucial during prolonged GNSS signal loss scenarios.true, false
enable_rolling_calibrationIf this parameter is set to true, the fusion algorithm will use a rough VIO / GNSS calibration at first and then refine it. This allows you to quickly get a fused positiontrue, false
enable_translation_uncertainty_targetWhen this parameter is enabled (set to true), the calibration process between GNSS and VIO accounts for the uncertainty in the determined translation, thereby facilitating the calibration termination. The maximum allowable uncertainty is controlled by the ‘target_translation_uncertainty’ parametertrue, false
gnss_vio_reinit_thresholdDetermines the threshold for GNSS/VIO reinitialization. If the fused position deviates out of the region defined by the product of the GNSS covariance and the gnss_vio_reinit_threshold, a reinitialization will be triggereddouble
target_translation_uncertaintyDefines the target translation uncertainty at which the calibration process between GNSS and VIO concludes. By default, the threshold is set at 10 centimeters.double
target_yaw_uncertaintyDefines the target yaw uncertainty at which the calibration process between GNSS and VIO concludes. The unit of this parameter is in radian. By default, the threshold is set at 0.1 radiansdouble

Mapping parameters

Namespace: mapping

common_stereo.yaml

ParameterDescriptionValue
mapping_enabledEnable/disable the mapping moduletrue, false
resolutionResolution of the fused point cloud [0.01, 0.2]double, default=0.05
max_mapping_rangeMaximum depth range while mapping in meters (-1 for automatic calculation) [2.0, 20.0]double
fused_pointcloud_freq*Publishing frequency (Hz) of the 3D map as fused point clouddouble
clicked_point_topicTopic published by RViz when a point of the cloud is clicked. Used for plane detectionstring, default "/clicked_point"
pd_max_distance_thresholdPlane detection: controls the spread of plane by checking the position differencedouble
pd_normal_similarity_thresholdPlane detection: controls the spread of plane by checking the angle differencedouble
publish_det_planeEnable publishing of the detected planestrue, false

* Dynamic parameter

Sensors parameters

Namespace: sensors

common_stereo.yaml

ParameterDescriptionValue
publish_imu_tfEnable publishing of transform imu_link -> left_camera_frame.true, false
sensors_image_syncSynchronize Sensors messages with latest published video/depth messagetrue, false
sensors_pub_rate*Frequency of publishing of sensors data. MAX: 400. - MIN: grab ratedouble
imu_sampling_rate[ZED SDK >= 5.6] IMU sampling rate. DEFAULT is the camera default (200 Hz on ZED X cameras); HZ_400 gives twice the IMU samples: set sensors_pub_rate to 400.0 to publish all of them. ZED X family only, with ZED X driver 1.5.0 or newerstring, ‘DEFAULT’, ‘HZ_200’, ‘HZ_400’
imu_accelerometer_range[ZED SDK >= 5.6] Accelerometer full-scale range. DEFAULT is the camera default (±12 g on ZED X cameras). A smaller range gives a finer resolution, a larger one avoids saturation under strong accelerations. ZED X family only, with ZED X driver 1.5.0 or newerstring, ‘DEFAULT’, ‘G_3’, ‘G_6’, ‘G_12’, ‘G_24’
imu_gyroscope_range[ZED SDK >= 5.6] Gyroscope full-scale range. DEFAULT is the camera default (±1000 deg/s on ZED X cameras). ZED X family only, with ZED X driver 1.5.0 or newerstring, ‘DEFAULT’, ‘DPS_250’, ‘DPS_500’, ‘DPS_1000’, ‘DPS_2000’
publish_imuEnable publishing of IMU datatrue, false
publish_imu_rawEnable publishing of raw IMU datatrue, false
publish_cam_imu_transfEnable publishing of transform between camera and IMU framestrue, false
publish_magEnable publishing of Magnetometer datatrue, false
publish_baroEnable publishing of Barometer datatrue, false
publish_tempEnable publishing of Temperature datatrue, false

* Dynamic parameter

Since v5.6.0 of the ROS 2 Wrapper, when the input does not provide the IMU data at the sensor rate, such as an SVO recorded with the legacy SVO v1 format, the node logs a single warning and does not publish the sensors topics. Set sensors_image_sync to true to publish the IMU data at the camera frame rate with such an input: the sensors messages are then published together with the video and depth messages, so only while a node subscribes to a video or depth topic.

Since v5.6.0 of the ROS 2 Wrapper, the IMU sampling rate and ranges actually used by the camera are logged at start-up. With a ZED X driver older than 1.5.0, or with another camera model, the IMU keeps its default configuration and the ZED SDK logs a warning.

Starting from v5.5.0 of the ROS 2 Wrapper the sensors stream is read independently of grab(), so the IMU publishing delay no longer depends on general.grab_frame_rate nor on general.grab_compute_capping_fps. Measured on a ZED X on an AGX Orin, the median publish delay dropped from 411 ms to 10.5 ms and the output rate went from 92.6 Hz to the requested 100.0 Hz.

sensors.sensors_pub_rate is now decimated against the measured sample interval instead of the rate the ZED SDK advertises, so it is honored on cameras whose IMU does not run at the advertised rate (with ZED SDK 5.5 or earlier, a ZED X One GS advertises 400 Hz but delivers 200 Hz), and cameras that report correctly produce exactly the requested rate.

Two consequences of the reduced CPU usage of the sensors thread: a new subscriber to a sensors topic can wait up to 250 ms for its first message, and a subscriber whose QoS is incompatible with the publisher no longer triggers publishing (it could never receive the data anyway).

Object detection parameters

Namespace: object_detection

common_stereo.yaml

ParameterDescriptionValue
od_enabledAutomatically enables Object Detection when the node startstrue, false
enable_trackingWhether the object detection system includes object tracking capabilities across a sequence of imagestrue, false
detection_modelDetection model. The three CUSTOM_* models run a user-provided ONNX file inside the ZED SDK and are configured in custom_object_detection.yaml; CUSTOM_BOX_OBJECTS_AUTODETECT picks YOLO-like or RF-DETR-like from the model outputsstring, ‘MULTI_CLASS_BOX_FAST’, ‘MULTI_CLASS_BOX_MEDIUM’, ‘MULTI_CLASS_BOX_ACCURATE’, ‘PERSON_HEAD_BOX_FAST’, ‘PERSON_HEAD_BOX_ACCURATE’, ‘CUSTOM_YOLOLIKE_BOX_OBJECTS’, ‘CUSTOM_RFDETRLIKE_BOX_OBJECTS’ [ZED SDK >= 5.5], ‘CUSTOM_BOX_OBJECTS_AUTODETECT’ [ZED SDK >= 5.5]
allow_reduced_precision_inferenceAllow inference to run at a lower precision to improve runtime and memory usagetrue, false
max_rangeDefines an upper depth range for detections [m]. The value cannot be greater than depth.max_depthdouble
prediction_timeoutDuring this time [sec], the object will have OK state even if it is not detected. Set this parameter to 0 to disable SDK predictionsdouble
filtering_modeDefines the filtering mode that should be applied to raw detectionsstring, ‘NONE’, ‘NMS3D’, ‘NMS3D_PER_CLASS’

object_detection.yaml

ParameterDescriptionValue
class.people.enabled*Enable detection of peopletrue, false
class.people.confidence_threshold*Minimum value of the detection confidence of peopledouble, [0,99]
class.vehicle.enabled*Enable detection of vehiclestrue, false
class.vehicle.confidence_threshold*Minimum value of the detection confidence of vehiclesdouble, [0,99]
class.bag.enabled*Enable detection of bagstrue, false
class.bag.confidence_threshold*Minimum value of the detection confidence of bagsdouble, [0,99]
class.animal.enabled*Enable detection of animalstrue, false
class.animal.confidence_threshold*Minimum value of the detection confidence of animalsdouble, [0,99]
class.electronics.enabled*Enable detection of electronicstrue, false
class.electronics.confidence_threshold*Minimum value of the detection confidence of electronicsdouble, [0,99]
class.fruit_vegetable.enabled*Enable detection of fruit and vegetablestrue, false
class.fruit_vegetable.confidence_threshold*Minimum value of the detection confidence of fruit and vegetablesdouble, [0,99]
class.sport.enabled*Enable detection of sportstrue, false
class.sport.confidence_threshold*Minimum value of the detection confidence of sportsdouble, [0,99]

* Dynamic parameter

custom_object_detection.yaml

ParameterDescriptionValue
object_detection.custom_onnx_filePath to the YOLO-like ONNX file for custom object detection directly performed by the ZED SDKstring
object_detection.custom_onnx_input_sizeResolution used with the YOLO-like ONNX file. For example, 512 means an input tensor ‘1x3x512x512’int
object_detection.custom_class_countNumber of classes in the custom ONNX file. For example, 80 for YOLOv8 trained on COCO datasetint

FOR EACH CLASS CREATE THE FOLLOWING PARAMETERS

ParameterDescriptionValue
object_detection.class_XXX.labelLabel of the custom class. The label must be unique and not used by other classesstring
object_detection.class_XXX.model_class_idClass ID of the object in the custom ONNX file (it is not required that this value matches the value in the ‘class_XXX’ identifier)int
object_detection.class_XXX.enabled*Enable/disable the detection of this classtrue, false
object_detection.class_XXX.confidence_threshold*Minimum value of the detection confidence of an object [0,99]double
object_detection.class_XXX.is_grounded*Provide hypothesis about the object movements (degrees of freedom or DoF) to improve the object trackingtrue, false
object_detection.class_XXX.is_static*Provide hypothesis about the object staticity to improve the object trackingtrue, false
object_detection.class_XXX.tracking_timeout*Maximum tracking time threshold (in seconds) before dropping the tracked object when unseen for this amount of timedouble
object_detection.class_XXX.tracking_max_dist*Maximum tracking distance threshold (in meters) before dropping the tracked object when unseen for this amount of meters. Only valid for static objectdouble
object_detection.class_XXX.max_box_width_normalized*Maximum allowed width normalized to the image sizedouble
object_detection.class_XXX.min_box_width_normalized*Minimum allowed width normalized to the image sizedouble
object_detection.class_XXX.max_box_height_normalized*Maximum allowed height normalized to the image sizedouble
object_detection.class_XXX.min_box_height_normalized*Minimum allowed height normalized to the image sizedouble
object_detection.class_XXX.max_box_width_meters*Maximum allowed 3D widthdouble
object_detection.class_XXX.min_box_width_meters*Minimum allowed 3D widthdouble
object_detection.class_XXX.max_box_height_meters*Maximum allowed 3D heightdouble
object_detection.class_XXX.min_box_height_meters*Minimum allowed 3D heightdouble
object_detection.class_XXX.object_acceleration_presetObject acceleration presetstring, ‘DEFAULT’, ‘LOW’, ‘MEDIUM’, ‘HIGH’
object_detection.class_XXX.max_allowed_acceleration*If set with a different value from the default [100000], this value takes precedence over the selected preset, allowing for a custom maximum acceleration. Unit is m/s^2double
object_detection.class_XXX.velocity_smoothing_factor*Controls the smoothing of the velocity estimationdouble, [0.0, 1.0], default 0.5
object_detection.class_XXX.min_velocity_threshold*Threshold to force an object’s velocity to zerodouble, default 0.2 m/s
object_detection.class_XXX.prediction_timeout_s*Duration to keep predicting a track’s position after occlusiondouble, default 0.5 s
object_detection.class_XXX.min_confirmation_time_s*Minimum confirmation time required to validate a trackdouble, default 0.05 s

* Dynamic parameter

Body Tracking parameters

Namespace: body_tracking

common_stereo.yaml

ParameterDescriptionValue
bt_enabledTrue to enable Body Trackingtrue, false
modelDetection modelstring, ‘HUMAN_BODY_FAST’, ‘HUMAN_BODY_MEDIUM’, ‘HUMAN_BODY_ACCURATE’
body_formatSkeleton format. Default: BODY_34. Only BODY_18 and BODY_34 have a GEN_2 network; BODY_34 requires enable_body_fitting to be true, and the ZED SDK forces it on otherwisestring, ‘BODY_18’,‘BODY_34’,‘BODY_38’
model_gen[ZED SDK >= 5.5] Body tracking network generation. GEN_2 (default) is bottom-up: more robust in crowded scenes and to unusual body orientations, detects people further from the camera, and is up to 18% faster and 14% more accurate than GEN_1. Overridden by the ZED_SDK_BODY_TRACKING_MODEL_GENERATION environment variable when setstring, ‘GEN_1’, ‘GEN_2’
allow_reduced_precision_inferenceAllow inference to run at a lower precision to improve runtime and memory usage. It has no effect on GEN_2, which always runs in FP16true, false
max_rangeDefines an upper depth range for detections [m]double
body_kp_selectionDefines the body selection output by the ZED SDK when retrieveBodies is calledstring, ‘FULL’, ‘UPPER_BODY’
enable_body_fittingDefines if the body fitting will be applied. Default: truetrue, false
enable_trackingDefines if the body tracking will track bodies across the image flowtrue, false
prediction_timeout_sDuring this time [sec], the skeleton will have OK state even if it is not detected. Set this parameter to 0 to disable SDK predictionsdouble
confidence_threshold*Minimum value of the detection confidence of skeleton key pointsdouble, [0,99]
minimum_keypoints_threshold*Minimum number of skeleton key points to be detected for a valid skeletonint

* Dynamic parameter

Starting from v5.5.0 of the ROS 2 Wrapper the default body_tracking.body_format is BODY_34 instead of BODY_38, and body_tracking.enable_body_fitting defaults to true. Only BODY_18 and BODY_34 have a GEN_2 network, so the previous default silently fell back to the older GEN_1. Skeletons published on ~/body_trk/skeletons now carry 34 keypoints instead of 38: set body_format back to BODY_38 to keep the previous topology, at the cost of running GEN_1.

Advanced parameters

Namespace: advanced

These are advanced threading parameters. Do not modify them unless you are confident of what you are doing. The SCHED_FIFO and SCHED_RR scheduling policies require root privileges, the CAP_SYS_NICE capability, or an RTPRIO limit (ulimit -r) at least equal to the requested priority.

With GMSL2 cameras, ZED SDK >= 5.6 and ZED X driver >= 1.5.0 run the capture and sensors threads of the camera at SCHED_RR priority 5. The ZED SDK installer grants this priority to the members of the zed group, from their next login. A node started by a systemd service that does not run as root needs LimitRTPRIO=5 in the [Service] section, and a Docker container needs --ulimit rtprio=5 (or --cap-add=SYS_NICE). Otherwise, these threads run at normal priority and the ZED SDK logs how to allow it. Giving the node threads a higher real-time priority than 5 can delay the camera capture. Reference: sched(7).

common_stereo.yaml

ParameterDescriptionValue
change_thread_priorityMaster switch for the other advanced parameters: the thread scheduling policy and the priorities below are applied only when this is truetrue, false
thread_sched_policyThread scheduling policystring, ‘SCHED_OTHER’, ‘SCHED_BATCH’, ‘SCHED_FIFO’, ‘SCHED_RR’
thread_grab_priorityPriority of the grab thread (only with ‘SCHED_FIFO’ and ‘SCHED_RR’)int, [1 (LOW) - 99 (HIGH)]
thread_sensor_priorityPriority of the sensors thread (only with ‘SCHED_FIFO’ and ‘SCHED_RR’)int, [1 (LOW) - 99 (HIGH)]
thread_pointcloud_priorityPriority of the point cloud thread (only with ‘SCHED_FIFO’ and ‘SCHED_RR’)int, [1 (LOW) - 99 (HIGH)]

Debug parameters

Namespace: debug

common_stereo.yaml

ParameterDescriptionValue
sdk_verboseSet the verbose level of the ZED SDKint, 0 -> disable
sdk_verbose_log_filePath to the file where the ZED SDK will log its messages. If empty, no file will be created. The log level can be set using the sdk_verbose parameter.string
use_pub_timestampsUse the current ROS time for the message timestamp instead of the camera timestamp. This is useful to test data communication latency.true, false
debug_commonEnable general debug log outputstrue, false
debug_dyn_paramsEnable dynamic parameters update debug log outputstrue, false
debug_grabEnable grab function/thread debug log outputstrue, false
debug_simEnable simulation debug log outputstrue, false
debug_video_depthEnable video/depth debug log outputstrue, false
debug_camera_controlsEnable camera controls debug log outputstrue, false
debug_point_cloudEnable point cloud debug log outputstrue, false
debug_tfEnable TF broadcasting debug log outputstrue, false
debug_positional_trackingEnable positional tracking debug log outputstrue, false
debug_gnssEnable GNSS fusion debug log outputstrue, false
debug_sensorsEnable sensors debug log outputstrue, false
debug_mappingEnable mapping debug log outputstrue, false
debug_terrain_mappingEnable terrain mapping debug log outputstrue, false
debug_object_detectionEnable object detection debug log outputstrue, false
debug_body_trackingEnable body tracking debug log outputstrue, false
debug_roiEnable region of interest debug log outputstrue, false
debug_streamingEnable streaming debug log outputstrue, false
debug_advancedEnable advanced debug log outputstrue, false
debug_nitrosEnable Nitros debug log outputstrue, false
disable_nitrosIf available, disable NITROS usage for debugging and testing purposestrue, false

Dynamic parameters

The ZED node lets you reconfigure many parameters dynamically during the execution of the node. All the dynamic parameters are indicated with a * in the list above and with the tag [DYNAMIC] in the comments of the YAML files.

You can set the parameters using the CLI command ros2 param set, e.g.:

ros2 param set /zed/zed_node depth.depth_confidence 80

If the parameter is set successfully, you will get a confirmation message:

Set parameter successful

In the case you tried to set a parameter that’s not dynamically reconfigurable, or you specified an invalid value, you will get this type of error:

ros2 param set /zed/zed_node depth.depth_confidence 150
Setting parameter failed: Parameter {depth.depth_confidence} doesn't comply with integer range.

A video setting accepted by the node parameter can still be refused by the camera, when the camera model or the input (SVO, stream, simulation) does not support it, or when the camera rejects the value. Since v5.6.0 of the ROS 2 Wrapper [ZED SDK >= 5.6], the node logs a single warning naming the setting and the reason, and does not retry it until the video parameters change again. Older ZED SDK versions reported such settings as applied. To stop the node from setting video.exposure_compensation or video.ae_antibanding on a camera that refuses it, set the parameter to -1: the camera keeps its own value. The default configuration of the Virtual Stereo Camera (virtual.yaml) already does so for both parameters.

You can also use the Configuration -> Dynamic Reconfigure plugin of the rqt tool to dynamically set parameters by using a graphic interface.

image

Transform frames

The ZED ROS 2 wrapper broadcasts multiple coordinate frames that each provides information about the camera’s position and orientation. If needed, the reference frames can be changed in the launch file.

  • <camera_name>_camera_link is the current position and orientation of the ZED base center. It corresponds to the bottom central fixing hole.
  • <camera_name>_camera_center is the current position and orientation of ZED middle baseline, determined by visual odometry and the tracking module.
  • <camera_name>_right_camera_frame is the position and orientation of the ZED’s right camera.
  • <camera_name>_right_camera_frame_optical is the position and orientation of the ZED’s right camera optical frame.
  • <camera_name>_left_camera_frame is the position and orientation of the ZED’s left camera.
  • <camera_name>_left_camera_frame_optical is the position and orientation of the ZED’s left camera optical frame.
  • <camera_name>_imu_link is the origin of the inertial data frame (not available with ZED).
  • <camera_name>_mag_link is the origin of the magnetometer frame (ZED2/ZED2i only).
  • <camera_name>_baro_link is the origin of the barometer frame (ZED2/ZED2i only).
  • <camera_name>_temp_left_link is the origin of the left temperature frame (ZED2/ZED2i only).
  • <camera_name>_temp_right_link is the origin of the right temperature frame (ZED2/ZED2i only).

For RViz 2 compatibility, the root frame map_frame is called map. The TF tree generated by the zed_wrapper reflects the standard described in REP105. The odometry frame is updated using only the “visual inertial odometry” information (loop closure is not applied). The map frame is updated using the Positional Tracking algorithm provided by the StereoLabs SDK, fusing the inertial information from the IMU sensor, and applying loop closure information.

map_frame (`map`)
└─odometry_frame (`odom`)
└─camera_link (`<camera_name>_camera_link`)
└─camera_frame (`<camera_name>_camera_center`)
└─left_camera_frame (`<camera_name>_left_camera_frame`)
| └─left_camera_frame_optical (`<camera_name>_left_camera_frame_optical`)
| └─imu_frame (`<camera_name>_imu_link`)
└─right_camera_frame (`<camera_name>_right_camera_frame`)
└─right_camera_frame_optical (`<camera_name>_right_camera_frame_optical`)

Static transforms and positional tracking

The node publishes the frames of the sensors, from <camera_name>_camera_center down. The transform from the base frame <camera_name>_camera_link to <camera_name>_camera_center comes from the ZED URDF, published by the robot_state_publisher node that zed_camera.launch.py starts with publish_urdf:=true (default). The positional tracking needs it to refer the camera pose to the base frame.

Since v5.6.0 of the ROS 2 Wrapper, when no node publishes this transform, for example with publish_urdf:=false or when the component is loaded in your own application without the ZED URDF, the node waits for it for 10 seconds when it starts the positional tracking, then logs a warning and starts the positional tracking without it:

  • the pose and the odometry describe <camera_name>_camera_center instead of <camera_name>_camera_link. The initial pose (pos_tracking.initial_base_pose) is applied to <camera_name>_left_camera_frame.
  • the ~/odom messages carry <camera_name>_camera_center as child_frame_id.
  • the odom -> <camera_name>_camera_link transform is not published, so the camera frames are not connected to odom and map.

As soon as the transform is available, the node logs it, the pose and the odometry move to <camera_name>_camera_link with no jump of the camera position, and the odom -> <camera_name>_camera_link transform is published. To get the full TF tree without the ZED URDF, publish the static transform from <camera_name>_camera_link to <camera_name>_camera_center yourself.

The same applies when the positional tracking is started for another module, such as depth.depth_stabilization.

When the Geo Tracking module is enabled the ZED Node TF Tree reflects the following diagram:

image

Services

The ZED node provides the following services:

  • ~/reset_odometry (std_srvs/srv/Trigger): Resets the odometry values eliminating the drift due to the Visual Odometry algorithm. The new odometry value is set to the latest camera pose received from the tracking algorithm. Only available if positional tracking is enabled.
  • ~/reset_pos_tracking (std_srvs/srv/Trigger): Restarts the Tracking algorithm setting the initial pose to the value available in the param server or to the latest pose set with the service set_pose. Only available if positional tracking is enabled.
  • ~/set_pose (zed_msgs/srv/SetPose): Restarts the Tracking algorithm setting the initial pose of the camera to the value passed as vector parameter -> [X, Y, Z, R, P, Y]. Only available if positional tracking is enabled.
  • ~/save_area_memory (zed_msgs/srv/SaveAreaMemory): Saves the Area Memory (.area file) of the current session so that it can be reloaded later with the pos_tracking.area_file_path parameter. The file path is passed in the request; if it is empty the value of pos_tracking.area_file_path is used. Only available if positional tracking is enabled.
  • ~/enable_depth (std_srvs/srv/SetBool): enable/disable the depth processing at runtime. Not available if depth.depth_mode is NONE.
  • ~/enable_obj_det (std_srvs/srv/SetBool): enable/disable object detection.
  • ~/enable_body_trk (std_srvs/srv/SetBool): enable/disable body tracking.
  • ~/enable_mapping (std_srvs/srv/SetBool): enable/disable spatial mapping.
  • ~/enable_streaming (std_srvs/srv/SetBool): enable/disable a local streaming server.
  • ~/start_svo_rec (zed_msgs/srv/StartSvoRec): Start recording an SVO file. If no filename is provided the default zed.svo2 is used. A filename without a path is saved in the working directory of the node process.
  • ~/stop_svo_rec (std_srvs/srv/Trigger): Stop an active SVO recording.
  • ~/pause_svo_rec (std_srvs/srv/SetBool): Pause (data: true) and resume (data: false) an active SVO recording without closing the file. It returns success: false when no recording is active, and the diagnostic reports SVO Recording: PAUSED while paused.
  • ~/toggle_svo_pause (std_srvs/srv/Trigger): set/reset SVO playing pause. Only available if svo.svo_realtime is false and if an SVO has been chosen as input source.
  • ~/set_svo_frame (zed_msgs/srv/SetSvoFrame): Set the SVO playback frame to the specified value. Only available if an SVO has been chosen as input source.
  • ~/set_roi (zed_msgs/srv/SetROI): Set the Region of Interest to the described polygon.
  • ~/reset_roi (std_srvs/srv/Trigger): Reset the Region of Interest to the full image frame.
  • ~/toLL (robot_localization/srv/ToLL): converts from map coordinate frame to Latitude/Longitude. Only available if GNSS fusion is enabled.
  • ~/fromLL (robot_localization/srv/FromLL): converts from Latitude/Longitude to map coordinate frame. Only available if GNSS fusion is enabled.
  • ~/reboot_camera (std_srvs/srv/Trigger): Reboots the camera without restarting the node: the node closes the camera, reboots it, reopens it and resumes publishing. The call returns when the camera is back, after 5 to 15 seconds, or fails if the camera does not come back within general.camera_timeout_sec x general.camera_max_reconnect seconds, and the node then stops as after a disconnection (see When the node stops). Available for the GMSL cameras, the ZED 2 and the ZED 2i with a live input, and not while GNSS fusion is enabled.

When positional tracking is disabled (pos_tracking.pos_tracking_enabled: false) the positional tracking and the spatial mapping services are not advertised at all.

Since v5.6.0 of the ROS 2 Wrapper, ~/reboot_camera reboots the camera from a running node. While it runs no data is published and the other services of the node wait. After the reboot the node starts again the modules that were enabled: the positional tracking restarts from the last valid pose, so odom and map stay continuous (with pos_tracking.floor_alignment, the floor is not searched again because that pose is already aligned to it), and object detection, body tracking, spatial mapping (with an empty map), the streaming server and the Region of Interest are enabled again. An active SVO recording is stopped. The diagnostic reports Camera rebooting while it runs, then the Reboots entry gives the number of reboots with the time and result of the last one. Rebooting a GMSL camera restarts the GMSL capture of the device: the other GMSL cameras can stop for a few seconds, and their nodes wait for them as for a disconnection.

Services can be called using the rqt graphical tool by enabling the plugin Plugins -> Services -> Service caller.

It is possible to call a service also by using the CLI command ros2 service call.

For example, to start Object Detection:

ros2 service call /zed/zed_node/enable_obj_det std_srvs/srv/SetBool "{data: true}"

and the service server will reply:

waiting for service to become available...
requester: making request: std_srvs.srv.SetBool_Request(data=True)
response:
std_srvs.srv.SetBool_Response(success=True, message='Object Detection started')

Assigning a GPU to a camera

To improve performance, you can specify the gpu_id of the graphic card that will be used for the depth computation in the common_stereo.yaml configuration file. The default value (-1) will select the GPU with the highest number of CUDA cores. When using multiple ZEDs, you can assign each camera to a GPU to increase performance.

Node Diagnostic

The ZED ROS 2 node publishes useful diagnostic information aggregated into the /diagnostics topic using the diagnostic_updater package.

Diagnostic information can be analyzed and parsed by using ROS 2 tools, like for example the Runtime Monitor plugin of rqt.

Available information:

  • Uptime
  • Camera disconnections: count and time of the last reconnection (since v5.6.0 of the ROS 2 Wrapper)
  • Grab frequency and processing time
  • Frame drop rate
  • Input mode
  • Video and Depth publishing rate and processing time
  • Depth Mode
  • Point Cloud publishing rate and processing time
  • GNSS Fusion status
  • Odometry and Localization status
  • Pose confidence: the [0,100] quality metric of the positional tracking. It requires ZED SDK >= 5.4 and POSITIONAL_TRACKING_MODE::GEN_3, and is reported as N/A otherwise
  • TF broadcasting rate
  • Object Detection status and publishing rate
  • Body Tracking status and publishing rate
  • IMU TF broadcasting rate
  • Sensor data publishing rate
  • Camera Internal temperatures
  • Scene Illuminance (ZED X family)
  • Camera health status: image quality, lighting, depth reliability, motion sensors reliability and, with ZED SDK >= 5.5 and zed_msgs v5.5, duplicated images
  • SVO playback status
  • SVO recording status, including the PAUSED state
  • Streaming server status

The diagnostic goes to WARN and names the issues while any camera health check is raised. It reports DISABLED when general.enable_image_validity_check is off.

image