Multi-Camera Tracking
Since ZED SDK 5.6, several ZED and ZED One cameras mounted on the same rigid body can be tracked together, as one rig, with a single shared map. This multi-camera SLAM runs through the Sensors API (sl::Sensors), with PositionalTrackingSensorsParameters::multi_sensor_mode set to MULTI_SENSOR_TRACKING_MODE::SINGLE_BODY.
The Sensors API is in beta: its interfaces and behavior may change in future ZED SDK releases. Multi-camera tracking is available in the C++ and Python APIs.
Tracking Modes
Sensors::getMultiSensorTrackingMode() returns the mode the tracking actually runs in.
Requirements
POSITIONAL_TRACKING_MODE::GEN_3.- At least one ZED stereo camera, and up to 8 cameras in total. LiDARs cannot be part of the rig.
- All cameras at the same resolution and frame rate, sharing one CUDA context (see
InitSensorsParameters::sdk_cuda_ctx). - Hardware-synchronized cameras, whose simultaneous images are captured within 1.5 ms of each other. This is checked on the first images: if it fails, an error is logged and the pose stays
POSITIONAL_TRACKING_STATE::UNAVAILABLE. See Frame synchronization. - The pose of each camera on the rig, set with
Sensors::setSensorPose()before enabling the tracking. It is expressed in the BASELINK frame of the rig, in thecoordinate_unitsandcoordinate_systemofInitSensorsParameters.
enablePositionalTracking() returns SENSORS_ERROR_CODE::INVALID_FUNCTION_PARAMETERS when the mode is not GEN_3, when the rig contains a LiDAR, no ZED stereo camera or more than 8 cameras, when the cameras do not share one resolution, frame rate and CUDA context, or when tracking_camera_id is not a ZED stereo camera of the rig. The hardware synchronization is checked on the first images instead, as described above.
How It Works
- One ZED stereo camera, set by
PositionalTrackingSensorsParameters::tracking_camera_id(by default the first ZED stereo camera added), drives the tracking of the rig: its IMU is the one fused, and its pose places the rig in the world. - Each ZED One adds its images to the shared map and moves with the rig through its sensor pose.
- The tracking of each sensor starts at its sensor pose, so the WORLD frame matches the BASELINK frame at start, unless you set
initial_world_transforms. InSINGLE_BODYmode, only theinitial_world_transformsentry of the tracking camera is used. getPosition()inSENSORS_REFERENCE_FRAME::WORLDreturns the pose of a sensor in the world frame.SENSORS_REFERENCE_FRAME::SENSORreturns its motion since the previous frame in its own axes, andSENSORS_REFERENCE_FRAME::BASELINKthe same motion in the BASELINK axes.saveAreaMap()saves the shared map, with the position of each camera on the rig. Area files in this format cannot be loaded by ZED SDK 5.5 or earlier.
Using the API
Using a Motion Prior
Sensors::ingestMotionPrior() feeds your own motion source to the Sensors API tracking, like Camera::ingestMotionPrior(). The difference is RelativePoseData::sensor_transform: it is the pose of your source in the BASELINK frame, not in the frame of a camera.
- In
SINGLE_BODYmode, the motion is given to the rig: call it once per sample, with the default identifier or any camera of the rig. - In
PER_SENSORmode, the motion is given to the camera named bysensors_idonly: call it for each camera that should use the source.
Sample
ZED SDK 5.6 adds a C++ sample for this mode. It runs from live cameras, SVO files or streams listed in a YAML file and shows the rig, its path and the shared map in 3D. See the Positional Tracking samples.

