> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.stereolabs.com/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.stereolabs.com/_mcp/server.

# Using ZED with ROS 2 and Isaac Sim

> **Warning**
>
> This is documentation for previous versions of the **Isaac Sim ZED Extension \< 4.0.0**.
>
> For the most recent version, please refer to [this page](/docs/integrations/isaac-sim/)

This tutorial guides you to configure the [*Isaac Sim ROS 2 Bridge*](https://docs.isaacsim.omniverse.nvidia.com/latest/installation/install_ros.html#enabling-the-ros-bridge-extension) to send simulated information to a [*ZED ROS 2 Wrapper*](https://github.com/stereolabs/zed-ros2-wrapper) node.

## Install the dependencies

> **Note**
>
> The installation guide is valid only for Ubuntu. For other operating systems, please refer to the [official NVIDIA® documentation](https://docs.isaacsim.omniverse.nvidia.com/latest/installation/install_ros.html).

The Isaac Sim ROS 2 Bridge requires the *Vision Messages* ROS 2 package to run. You can install it by using the command

```bash
sudo apt install ros-humble-vision-msgs
```

If this dependency is not installed the *ROS 2 Bridge* extension will not start, and you will get an error message similar to this in the Isaac Sim log terminal:

```bash
2023-11-24 16:04:22 [73,377ms] [Error] [omni.isaac.ros2_bridge.plugin] Could not load ROS2 Bridge due to missing library dependencies, please make sure your sourced ROS2 workspace has the correct packages/libraries installed
[73.388s] To use the internal libraries included with the extension please set: 
RMW_IMPLEMENTATION=rmw_fastrtps_cpp
LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/home/<user>/.local/share/ov/pkg/isaac_sim-2023.1.0-hotfix.1/exts/omni.isaac.ros2_bridge/humble/lib
Before starting Isaac Sim
2023-11-24 16:04:22 [73,377ms] [Error] [omni.isaac.ros2_bridge.scripts.extension] ROS2 Bridge startup failed
```

**IMPORTANT**: do **not** add the new path to `LD_LIBRARY_PATH` in `~/.bashrc` as suggested, otherwise you will create conflicts between libraries that will not allow you to start `rviz2` and other ROS 2 tools.

For additional details, we recommend reading the [official NVIDIA® documentation](https://docs.isaacsim.omniverse.nvidia.com/latest/installation/install_ros.html).

## Setup the environment variables for ROS 2

Ensure that the ROS environment is sourced in the terminal or in your `~/.bashrc` file before starting *Isaac Sim*:

```bash
source /opt/ros/humble/setup.bash
```

> **Note**
>
> If you want to launch *Isaac Sim* by using the *Omniverse Launcher*, it is required that you add the line `source /opt/ros/humble/setup.bash` to `~/.bashrc` to automatically configure the ROS 2 environment variables each time you open a terminal console. Otherwise, you must launch that command in the terminal console before manually starting *Isaac Sim* with the command `isaac-sim.sh` which is located in the folder `~/.local/share/ov/pkg/isaac_sim-<version>/`.

## Initialize the simulator

First of all, remember to select the correct `ROS Bridge Extension` when starting *Isaac Sim*: **omni.isaac.ros2\_bridge**

![image](/_fern-img/6f950e37369971c5ddea1d09f5de9619699ff6e4709f90a6c867f5bdca8acd48.webp)

### Enable the ROS 2 Bridge Extension

When Isaac Sim is ready you must enable the `ROS2 Bridge` extension.

* Open the Extension view from the menu, **Window -> Extensions**.
* Search for `ROS2 Bridge` by using the search form in the top left.
* Click the `DISABLED` switch to set it to `ENABLED`.
* \[Optional] Flag the `AUTOLOAD` option to avoid performing these initialization steps each time you start Isaac Sim.

![image](/_fern-img/06c02725fe8e43f2797182a212f3cc51dd205100a35ce4011d84c278f1cce52f.webp)

### Enable the ZED Extension

The ZED Camera Extension is required to send the simulated ZED images to the [ZED ROS 2 Wrapper](https://github.com/stereolabs/zed-ros2-wrapper).

* Open the Extension view from the menu, **Window -> Extensions**.
* Search for "ZED" by using the search form in the top left.
  * Select `THIRD PARTY` if you have installed the extension from the GitHub repository (see [here](/docs/integrations/isaac-sim/old-version-of-the-zed-extension-for-isaac-sim/setting-up-the-zed-in-isaac-sim#activate-the-zed-camera-extension) for adding the extension if it's not available).
* Click the `DISABLED` switch to set it to `ENABLED`.
* \[Optional] Flag the `AUTOLOAD` option to avoid performing these initialization steps each time you start Isaac Sim.

![image](/_fern-img/e6eea5fd1198ea7808d431fe0d8d0d7a22402a42d7f9bdca985a09db1eae538d.webp)

## Add a scenario to the Stage

The ZED camera is a visual sensor, so it's important to add an environment that is rich in visual features for a good simulation experience.

Isaac Sim provides ready environments to be used, from **Create -> Isaac -> Environments** you can select for example the environment named `Small Warehouse with Multiple Shelves`.

If you are using [Nucleus](/docs/integrations/isaac-sim/old-version-of-the-zed-extension-for-isaac-sim/get-started-with-isaac-sim#install-nvidia-isaac-sim) and you created a local server, a web page will open with a form to insert some user credentials. If it's the first time that you use it, you must create a new user, and then use the new credentials to login to the server.

Wait for the simulator to load the environment. This task can take a while, according to the dimension of the environment that you selected.

When all the objects of the scene are finally loaded, the simulator will display the selected environment from the default point of view of the viewport camera.

![image](/_fern-img/22ace0dbdf6052138507d827ae1c5cd03f781c1b0e2dab6223dd58ef0c21a7a1.webp)

## Add a ZED camera to the scene

The ZED X model will be available in the **Isaac Assets** tab on the bottom panel, with the latest versions of the Isaac Sim app.

If you cannot find the ZED X Model, then you can manually add it by cloning the [GitHub repository](https://github.com/stereolabs/zed-isaac-sim):

```bash
cd <your_workspace>
git clone git@github.com:stereolabs/zed-isaac-sim.git
```

![image](/_fern-img/f8a0f9b1d25380cf7370ebd98f791d94fde196db8b01744653c260a08d72259e.webp)

Select the model **ZED\_X.usd** and drag it in the *Viewport* to place it in the position that you prefer.

A new prim named `ZED_X` will appear in the `Stage` view as a child of `World`.

![image](/_fern-img/f8db54934640c7965c05fc239fb4ecc70950a5fc4e66d8d203c997f82d566017.webp)

The `ZED_X` prim has an important XForm named `base_link`, this is the reference frame of the mounting point on the bottom of the camera.

### Enable the ZED camera

The data streaming is activated by using the Omnigraph node **ZED Camera streamer**.

* Create a new `Action Graph`, **Create -> Visual Scripting -> Action Graph**.
* Add a new node of type `On Playback Tick` to the graph. You can easily search for it by writing `Tick` in the search form.
* Type `ZED` in the search form and drag the `ZED Camera Streamer` node into the graph.
* Connect the `ExecIn` input of `ZED Camera Streamer` to the `Tick` output of the `On Playback Tick` node.

![image](/_fern-img/44a46c5cf7e294ee625b0d5780512b5255d08694e75d8c73c680d4ab42707e99.webp)

* Click on the `ZED Camera Streamer` in the graph and select the `Property` tab on the right.
* Click `+ Add Target` near `ZED Camera prim`.
* Select the root XForm named `ZED_X` and hit `Select`.

![image](/_fern-img/613cbf1f7b4cbd8224bae0b2f2aafd98644681309219f45f20b7c91123cbd4f0.webp)

## Publish simulation time to ROS 2

When working with simulated data, the ROS 2 nodes cannot rely on the system clock to timestamp the data accurately. Instead, they have to use the simulation time, which can be faster or slower than the real time.

To handle this behavior, Isaac Sim ROS 2 Bridge provides a method to publish messages to the `/clock` topic with simulation time data.

All the ROS 2 nodes have a parameter named `use_sim_time`. When this parameter is set to `true`, each ROS 2 node subscribes to the `/clock` topic, and uses the simulation time as the reference for time instead of the system time.

* Select the `Action Graph` tab in the bottom panel.
* Add a new node of type `Isaac Read Simulation Time`.
  * \[Optional] Click on the `Isaac Read Simulation Time`.
  * \[Optional] Select the `Properties` tab in the right panel.
  * \[Optional] Check the `Reset on stop` option to reset to zero the simulation time each time the simulation is stopped and restarted.
* Add a new node of type `ROS2 Publish Clock`.
* Connect the `Tick` output of the `On Playback Tick` node to the `ExecIn` input of `ROS2 Publish Clock`.
* Connect the `Simulation Time` output of the `Isaac Read Simulation Time` node to the `Time Stamp` input of `ROS2 Publish Clock`.

![image](/_fern-img/cf7e426afa58dd9583842a6fd1eabc3987f8c1fd87cc8064188fa980a3f23205.webp)

* Start the simulation by hitting the Play button on the left panel

![image](/_fern-img/98ca4e2568546feab14912accd42d8b04573ea62150d436aac5bb9d1659e338a.webp)

Now you can verify that the configuration is valid:

* open a Terminal console (`Ctrl + Alt + t`).
* enter the command `ros2 topic echo /clock`.
* verify that the `/clock` topic is correctly published:

```bash
ros2 topic echo /clock
clock:
  sec: 6
  nanosec: 900000359
---
clock:
  sec: 6
  nanosec: 916667027
---
clock:
  sec: 6
  nanosec: 933333694
---
clock:
  sec: 6
  nanosec: 950000362
---
[...]
```

* stop listening to the `/clock` topic with `Ctrl+c`.

## Start the ZED ROS 2 Wrapper node with simulated data

The ZED ROS 2 Wrapper enables connection to the ZED Isaac Sim Extension, publishing ZED data in ROS 2 as if a real camera is present.

### Prerequisites

The "ZED ROS 2 Wrapper" and the "ZED ROS 2 Examples" packages must be installed in the system.

Follow the relevant guides to install them if not already done:

* [Install the ZED ROS 2 Wrapper package](https://github.com/stereolabs/zed-ros2-wrapper#build-the-package)
* [Install the ZED ROS 2 Examples package](https://github.com/stereolabs/zed-ros2-examples#build-the-package)

### Launch the ZED ROS 2 node

Launch a standalone ZED ROS 2 node with simulated ZED data as input:

```bash
ros2 launch zed_wrapper zed_camera.launch.py camera_model:=zedx sim_mode:=true use_sim_time:=true
```

Launch options:

* \[Mandatory] `sim_mode`: start the ZED node in simulation mode if `true`.
* \[Mandatory] `use_sim_time`: force the node to wait for valid messages on the topic `/clock`, and use the simulation time as the reference.
* \[Optional] `sim_address`: set the address of the simulation server. Default is `127.0.0.1` and it's valid if the node runs on the same machine as the simulator.
* \[Optional] `sim_port`: set the port of the simulation server. It must match the value of the field `Streaming Port` of the properties of the `ZED camera streamer` Action Graph node. A different `Streaming Port` value for each camera is required in multi-camera simulations.

> `camera_model` is currently limited to `zedx`. We are working to support other models in the near future.

You can also start a preconfigured instance of `rviz2` to visualize all the information available in simulation by using the command:

```bash
ros2 launch zed_display_rviz2 display_zed_cam.launch.py camera_model:=zedx sim_mode:=true use_sim_time:=true
```

The `display_zed_cam.launch.py` launch file includes the `zed_camera.launch.py` launch file, so it provides the same parameters.

You can get the list of all the available launch parameters by using the `-s` launch option:

```bash
ros2 launch zed_wrapper zed_camera.launch.py -s
ros2 launch zed_display_rviz2 display_zed_cam.launch.py -s
```

Here's an example of `rviz2` running with the simulated information obtained by placing the ZED camera on a shelf of a simulated warehouse:

![image](/_fern-img/100639ac86a2e31ea9727cbc24fe804cf15160c6d6ceee2b39825de28ab80f91.webp)

![image](/_fern-img/d69a1178b74abf3d49b8ec00a46b8772572558031672e177957bf866789b7e19.webp)