Software IntegrationIsaac Sim

SDK-free depth capture in Isaac™ Sim

Open in ClaudeOpen in ChatGPT

The ZED Camera extension can capture RGB and depth from a simulated ZED directly inside NVIDIA® Isaac™ Sim, using the simulator’s renderer, with no ZED SDK and no streaming. This is useful for quick frame/depth capture, generating datasets, or working on a machine where the ZED SDK is not installed.

For how this compares to the streaming and Isaac Lab tensor paths, see Choosing a data path.

The depth captured here is Isaac™ Sim’s ground-truth renderer depth (distance_to_image_plane), not the ZED SDK’s stereo-matched depth. For genuine ZED SDK depth, use the streaming path.

ZED Depth OmniGraph node

The extension provides a ZED Depth node that captures RGB + depth each simulation tick. Add it to an Action Graph, connect an On Playback Tick node to its ExecIn, set its properties, and press Play. The ZED Depth node supports stereo camera models only; mono ZED X One cameras are not supported.

PropertyDescription
ZED Camera PrimThe ZED camera prim in the stage (e.g. /World/ZED_X).
Camera ModelSimulated stereo model: ZED_X, ZED_XM, ZED_X_Nano, ZED_M, or ZED_2i. Detected automatically from the connected ZED Camera Prim when possible.
Lens TypeWide or Narrow (options depend on the model). Detected automatically like the Camera Model.
ResolutionRender resolution: HD2K, HD1200, HD1080, HD720, SVGA, or VGA (filtered per model).
Enable SaveSave RGB (PNG) + depth (EXR) to disk each frame.
Output DirectoryDirectory to save frames. Required when Enable Save is on.

Standalone scripts

The extension also ships two capture scripts under exts/sl.sensor.camera/examples/isaac_sim/, both driven by the sl.sensor.camera.zed_depth.ZEDDepthCamera helper.

Standalone / headless (zed_depth_standalone.py)

Captures left RGB + depth for N frames, prints per-frame stats, and optionally saves PNG/EXR. It runs with no arguments: when no scene or camera is given, it builds a minimal ground + dome-light scene and spawns a ZED camera at a small default pose, so you get a usable depth capture out of the box.

$# Linux, zero-config: minimal scene + auto-spawned ZED_X
$./python.sh exts/sl.sensor.camera/examples/isaac_sim/zed_depth_standalone.py
$
$# Full control: load your own scene and/or use a camera already placed in it
$./python.sh exts/sl.sensor.camera/examples/isaac_sim/zed_depth_standalone.py \
>    --usd_path /path/to/scene.usd \
>    --camera_prim /World/ZED_X \
>    --camera_model ZED_X \
>    --resolution HD1200 \
>    --num_frames 10 \
>    --output_dir /tmp/zed_depth
$# Windows: use python.bat. Add --headless for no GUI.

Arguments (all optional): --usd_path (environment USD; omit to build a minimal scene), --camera_prim (an existing ZED camera prim; omit to spawn one), --camera_model, --resolution, --num_frames, --output_dir, --headless.

GUI Script Editor (zed_depth_example.py)

For interactive use inside the Isaac™ Sim GUI (extension enabled): load a scene with a ZED camera, paste the script into the Script Editor, edit the CONFIG block (CAMERA_PRIM_PATH, CAMERA_MODEL, RESOLUTION, CAPTURE_INTERVAL, SAVE_DIR), execute it, then press Play. RGB + depth are captured every CAPTURE_INTERVAL physics steps; press Stop to clean up.

Output

  • Console: per-capture shape and depth-range stats.
  • Files (when a save path is set): rgb_left_NNNNNN.png (left RGB) and depth_NNNNNN.exr (float32 depth in meters, 0 = invalid). For stereo models, rgb_right_NNNNNN.png (right RGB) is saved too; depth stays left-referenced.

See also