ZED Isaac™ ROS Integration Overview

NVIDIA® Isaac™ ROS (Robot Operating System) is a collection of NVIDIA® CUDA®-accelerated computing packages and AI models designed to streamline and expedite the development of advanced AI robotics applications.

This section explains how to use ZED cameras with NVIDIA® Isaac™ ROS 5 through the ZED ROS 2 Wrapper.

Supported versions

Isaac™ ROS 4.x is not covered by this documentation.

What changed with Isaac™ ROS 5

Isaac™ ROS 5 is a major release that changes how the ZED ROS 2 Wrapper integrates with the Isaac™ ROS packages:

• ROS 2 Lyrical replaces ROS 2 Humble. All the Isaac™ ROS packages, and the ZED ROS 2 Wrapper, must be built for ROS 2 Lyrical on Ubuntu 24.04.
• NITROS was removed. The isaac_ros_nitros and isaac_ros_managed_nitros packages no longer exist. NVIDIA replaced them with rosidl::Buffer and buffer backends, a native ROS 2 Lyrical feature that lets accelerated nodes exchange standard ROS 2 messages.
• Fast DDS or Zenoh is required. The buffer backends are supported by the rmw_fastrtps_cpp middleware, the default, and by rmw_zenoh_cpp v0.10.5 or newer. With other middleware implementations, such as rmw_cyclonedds_cpp, the data published by the Isaac™ ROS nodes does not reach the subscribers running in other processes. See Middleware requirement.
• The Isaac™ ROS CLI manages the environment. The isaac-ros command replaces the run_dev.sh script and supports three isolation modes: Docker, Virtual Environment, and Bare Metal.

What this means for ZED cameras

• The ZED ROS 2 Wrapper publishes standard ROS 2 messages (sensor_msgs/msg/Image, sensor_msgs/msg/CameraInfo, sensor_msgs/msg/PointCloud2, …) that Isaac™ ROS 5 nodes subscribe to directly, without any NITROS bridge. The ZED color images use the bgra8 encoding, or the bgr8 encoding when the video.enable_24bit_output parameter is true: when an Isaac™ ROS node requires a different encoding, such as rgb8, add the Isaac™ ROS image format converter as shown in the ZED test.
• Starting with v5.6.0, the ZED ROS 2 Wrapper supports the rosidl::Buffer message fields of ROS 2 Lyrical and publishes the images, the depth maps, the confidence maps, and the point clouds with the CUDA zero-copy transport: the nodes that accept the cuda buffer backend read the ZED data directly in GPU memory, without copies, and all the other nodes automatically receive standard CPU data. The general.buffer_backend parameter of the ZED Stereo Node and of the ZED Monocular Node controls this behavior: cuda (default) or cpu. Older versions publish the data from CPU memory.
• The .../nitros topics of the Isaac™ ROS 3 integration are no longer available.
• Intra-Process Communication (IPC) is enabled automatically because NITROS is not installed. You no longer need the enable_ipc:=false launch argument, and you can load the ZED node and the Isaac™ ROS nodes in the same container for efficient intra-process data exchange. IPC subscribers receive CPU data: to receive the ZED data in GPU memory, disable IPC as described in the CUDA zero-copy image tutorial.

Get started

1. Install NVIDIA® Isaac™ ROS 5 with the Docker or the Bare Metal installation path, and test the installation. Optionally, replace Fast DDS with Zenoh.
2. Use ZED with Isaac™ ROS: add the ZED SDK and the ZED ROS 2 Wrapper to your Isaac™ ROS environment, and verify that an Isaac™ ROS node processes the ZED images.
3. Set up the CUDA zero-copy transport: learn how the ZED data reach your accelerated nodes directly in GPU memory, then follow the tutorials for images and point clouds.
4. Detect objects with Isaac™ ROS: run the RT-DETR or YOLOv8 object detection on the ZED images, with the image kept in GPU memory from the ZED SDK to the TensorRT inference.