Skip to navigation

Video Recording

Open in ClaudeOpen in ChatGPT

The ZED SDK allows you to record large video datasets using H.264, H.265 or lossless compression. The ZED SDK uses StereoLabs’ SVO format to store videos along with additional metadata such as timestamps and sensor data.

When loading SVO files, the ZED API will behave as if a ZED was connected and a live feed was available. Every module of the ZED API will be available: depth, tracking, spatial mapping and more. To record SVO videos, you can use the ZED Explorer or ZED Studio applications in GUI or command-line mode or build your own recording app using the ZED API.

What an SVO File Contains

An SVO file is a recording of what the camera outputs, not of what the ZED SDK computes. It only contains:

  • The unrectified video stream, as output by the camera after its ISP: left and right images for a stereo camera, a single image for a monocular camera, encoded with the selected compression mode.
  • The sensor data that the camera actually provides, stored as metadata alongside the video: IMU (accelerometer and gyroscope) and temperature on the models that embed them, and magnetometer and barometer on the ZED 2 and ZED 2i only. Check the sensor availability table for your camera model. With SVO2, each sensor is recorded at its own full rate instead of the image frame rate.
  • The information required to replay the stream: camera model and serial number, resolution, frame rate, image and sensor timestamps, and any custom data you recorded yourself.

The calibration parameters of a real camera are not stored in the SVO file (since ZED SDK 5.6, only recordings of simulated cameras embed their calibration). They are resolved at playback time from the settings folder of the machine reading the file, using the serial number recorded in it, and downloaded from the StereoLabs servers if no local file is found. Replacing that SNxxxxxxx.conf file with a custom calibration therefore changes how an existing recording is rectified and how its depth is computed.

No processed data is ever written to the file. Rectified images, depth maps, point clouds, normal maps, confidence maps, camera poses, spatial maps, detected objects and bodies are not stored in an SVO file.

Everything Is Computed Live at Playback

While an SVO file is played back, the ZED SDK processes the raw frames exactly as it does with a live camera: rectification, depth estimation, positional tracking, spatial mapping, object and body detection all run on your machine as the file is read.

The configuration used during the recording session therefore does not constrain the playback session. A file recorded with DEPTH_MODE::NONE and no module enabled can be replayed with DEPTH_MODE::NEURAL_PLUS, positional tracking and object detection all enabled, and every one of them will produce results. The same applies to the depth range, the confidence and texture thresholds, the coordinate system and the unit system: they are chosen when you open the file, not when you record it.

Can I get depth from a file recorded with no depth mode? Yes, on a stereo camera. Depth is never stored in the file, it is always computed from the raw left and right images at playback time, so DEPTH_MODE is a playback choice. Monocular recordings (ZED X One) contain a single image stream and cannot produce depth.

Because everything is recomputed, the output of a given SVO file depends on the machine and the ZED SDK version used for the playback. Replaying the same file on a different GPU or after an SDK update can yield slightly different depth or tracking results.

What Cannot Be Changed at Playback

The recorded images have already been processed by the camera ISP, so everything applied before the frames were written is baked into the pixels:

  • Camera control settings (sl::VIDEO_SETTINGS): exposure, gain, white balance, gamma, saturation, sharpness, denoising, and so on. Calling setCameraSettings() has no effect during playback.
  • Resolution and frame rate, which are fixed when the file is created.

Changing any of these requires a new recording, which is why you should make sure the image settings are correct before starting a long capture session.

Compression Modes

SVO videos can be recorded using various compression modes. We provide both lossless and compressed modes to preserve image quality or reduce file size.

Compression ModeAverage Size (% of RAW)
LOSSLESS (PNG/ZSTD)42%
H.264 (AVCHD)1%
H.265 (HEVC)1%
H.264 LOSSLESS25%
H.265 LOSSLESS25%

Before ZED SDK 5.6, H264_LOSSLESS and H265_LOSSLESS recordings on Jetson with JetPack 6 were lossy, at 1 to 2% of the raw size. Since ZED SDK 5.6 they are near lossless, so the files are much larger than before: about 12 times larger with H265_LOSSLESS and 30 times with H264_LOSSLESS. For example, a ZED X Mini at HD1200 and 30 FPS writes about 75 MB/s with H265_LOSSLESS. Plan storage accordingly, and use LOSSLESS for images identical to the live ones.

Benefits of Hardware Encoding

For optimal performance, we recommend using the H.264 and H.265 recording modes. They have been designed to use the hardware-based encoder (referred to as NVENC) built into NVIDIA® graphics cards. With encoding offloaded to NVENC, the GPU and the CPU are free for other operations. For example, in a compute-heavy scenario, it is possible to record video at a full frame rate with minimal impact on the main application.

Encoding Quality

At a given bitrate, hardware encoding quality can vary depending on your GPU generation. Newer NVENC generations typically produce better quality than older ones at the same bitrate. Check the NVENC support matrix for the encoder generation of your GPU.

Encoding Presets

Starting with ZED SDK 5.3, you can select the speed/quality tradeoff of the hardware encoder with an encoding preset (sl::SVO_ENCODING_PRESET), set through the encoding_preset field of RecordingParameters. A faster preset lowers the load on the encoder, which helps when you record several cameras on the same GPU or when the encoder is shared with other tasks. A slower preset gives better image quality at the same bitrate.

PresetDescriptionNVENC preset (Desktop)V4L2 preset (Jetson)
DEFAULTEncoder default, used when no preset is setP4Encoder default
ULTRAFASTFastest encoding, lowest qualityP1ULTRAFAST
FASTFast encodingP2FAST
MEDIUMBalanced speed and qualityP3MEDIUM
SLOWSlowest encoding, highest qualityP5SLOW

The encoding preset only applies to the hardware encoded compression modes: H264, H265, H264_LOSSLESS and H265_LOSSLESS. It has no effect with the LOSSLESS (PNG/ZSTD) mode.

See Video Recording below for an example that sets a preset.

Using SVO2

Release 4.1 of the ZED SDK introduced the SVO2 file format, designed to store high-frequency data from the camera and to record custom data in order to store data from external sensors. This recording format is enabled by default in the ZED SDK starting from version 4.1.

  • High-frequency data: SVO(1) files would previously only record sensor data at the camera’s image frame rate (15-120 Hz). With SVO2, sensors are recorded at their respective frequency, enabling all ZED SDK algorithms which use high-frequency data, for example positional tracking with IMU fusion.
  • Custom Data: the SVO2 format exposes methods to record custom data. This data can be anything defined by the user, from metadata to label the specific SVO sequence, to data from external sensors, such as IMU, GPS, etc. All custom data is timestamped to be read alongside the original ZED data. More information on the Custom Data API here: C++ / Python / C# / C.

Please check out the custom data Recording and Playback samples on GitHub for more information.

Recording with Multiple Cameras

You can record videos with multiple cameras connected to the same PC. When using hardware encoding (H.264, H.265), make sure to check the NVENC support matrix or the Jetson one, which shows the maximum number of concurrent recording sessions that can be started on a single NVIDIA® GPU. You can also add multiple GPUs to a single server to increase the number of recording sessions with hardware encoding.

On NVIDIA® Jetson™, record each camera from its own process. Recording several cameras from the same application, for example with one sl::Camera per thread, can make the hardware encoder fail after a few minutes. All the cameras then stop, and the SVO files being recorded can be corrupted. We observed this on a Jetson AGX Orin recording four GMSL2 cameras at HD1200 and 30 fps with ZED SDK 5.6, while the same four recordings made from four separate processes completed without errors.

Start one instance of your application per camera, and select the camera by its serial number with init_parameters.input.setFromSerialNumber(<serial>) (see how to list connected cameras). If you use Docker, see one container per camera. With ROS 2, see multi-camera SVO recording.

Using the Recording API

Video Recording

To record SVO files, you need to enable the Recording module with enableRecording(). Specify an output file name (eg: output.svo), an SVO_COMPRESSION_MODE and optionally an encoding preset (ZED SDK 5.3 and later), then save each grabbed frame. SVO lets you record video and associated metadata (timestamp, IMU data and more if available).

// Create and open a ZED camera object
Camera zed;
InitParameters init_parameters;
ERROR_CODE err = zed.open(init_parameters);
if (err != ERROR_CODE::SUCCESS)
exit(-1);
// Enable recording with the filename specified in argument
String output_path(argv[1]);
RecordingParameters recordingParameters;
recordingParameters.compression_mode = SVO_COMPRESSION_MODE::H264;
// Optional (ZED SDK >= 5.3): favor encoding speed over quality
recordingParameters.encoding_preset = SVO_ENCODING_PRESET::FAST;
recordingParameters.video_filename = output_path;
err = zed.enableRecording(recordingParameters);
while (!exit_app) {
// Each new frame is added to the SVO file
zed.grab();
}
// Disable recording
zed.disableRecording();

Pausing and Stopping a Recording

With the ZED API, recording is never enabled by default: frames are only written to a file between enableRecording() and disableRecording(). You can stop or pause a recording at any time, and the camera keeps running in both cases.

  • Stop: disableRecording() stops the recording and closes the SVO file, which is then complete and ready to be played back. close() calls it automatically if a recording is still active.
  • Pause: pauseRecording(true) stops writing frames to the file without closing it, and pauseRecording(false) resumes writing to the same file. getRecordingStatus().is_paused tells you whether the recording is paused.
// Pause the recording, the camera keeps grabbing
zed.pauseRecording(true);
// Resume writing to the same SVO file
zed.pauseRecording(false);
// Check the recording state
RecordingStatus status = zed.getRecordingStatus();
if (status.is_recording && !status.is_paused) {
// Frames are being written to the file
}
// Stop the recording and close the file, the camera stays open
zed.disableRecording();

The other recording tools offer the same controls:

  • ROS 2: call the ~/stop_svo_rec service to stop the recording and ~/pause_svo_rec to pause and resume it, as described in Record and Replay Camera Data with ROS 2.
  • ZED Explorer in command-line mode (ZED_Explorer -o output.svo2): press Ctrl+C to stop the recording, or set the number of frames to record with -l <frames> so that it stops on its own. See the ZED Explorer command line options.

Always stop a recording with disableRecording(), close() or the stop command of your tool, so that the file is properly closed. If the application is killed or the machine loses power while recording, the file is left unfinished. The ZED SDK repairs such a file in place when open() reads it, and since ZED SDK 5.6 a file that cannot be repaired in place (read-only or full storage, file in use) still opens and plays back the frames recorded before the interruption. You can also write a repaired copy with ZED_SVO_Editor -repair input.svo2. Encrypted files cannot be repaired.

Video Playback

To play SVO files, simply add the file path as an argument in setFromSVOFile(). When loading SVO files, the ZED API will behave as if a ZED was connected and a live feed was available. Every module of the ZED API will be available: depth, tracking, spatial mapping and more, regardless of what was enabled during the recording session, as explained in What an SVO File Contains. When an SVO file is read entirely, END_OF_SVOFILE_REACHED error code is returned.

// Create a ZED camera object
Camera zed;
// Set SVO path for playback
String input_path(argv[1]);
InitParameters init_parameters;
init_parameters.input.setFromSVOFile(input_path);
// Open the ZED
ERROR_CODE err = zed.open(init_parameters);
sl::Mat svo_image;
while (!exit_app) {
ERROR_CODE grab_status = zed.grab();
if (grab_status == ERROR_CODE::SUCCESS) {
// Read side by side frames stored in the SVO
zed.retrieveImage(svo_image, VIEW::SIDE_BY_SIDE);
// Get frame count
int svo_position = zed.getSVOPosition();
}
else if (grab_status == ERROR_CODE::END_OF_SVOFILE_REACHED) {
std::cout << "SVO end has been reached. Looping back to first frame" << std::endl;
zed.setSVOPosition(0);
}
}

Since ZED SDK 5.6, SVO2 frames decode straight into GPU memory on desktop GPUs, DGX Spark and Jetson Thor, and Jetson Thor uses the GPU decoder instead of V4L2. Playback is up to 2 times faster and uses less CPU, with the same images. To restore the previous behavior, set the environment variable ZED_SDK_SVO_HOST_DECODE=1 (desktop) or ZED_SDK_SVO_V4L2_DECODER=1 (Jetson Thor).

Extracting IMU Data from an SVO2 File

SVO2 files store sensor data at each sensor’s own frequency, as described in Using SVO2. During playback, that data is read back with the same getSensorsData() call used with a live camera, and the TIME_REFERENCE you pass decides what you get:

  • TIME_REFERENCE::IMAGE returns the sample closest to the current video frame, so you get one sample per grab() call. This is what the example below does.
  • TIME_REFERENCE::CURRENT returns the most recent sample independently of grab(). Call it more often than you call grab() and compare timestamps to walk through the samples recorded between two frames, as explained in Retrieve New Sensor Data.
// Open the SVO file (same as standard playback)
Camera zed;
InitParameters init_parameters;
init_parameters.input.setFromSVOFile("recording.svo2");
zed.open(init_parameters);
SensorsData sensors_data;
while (!exit_app) {
if (zed.grab() == ERROR_CODE::SUCCESS) {
// Retrieve the IMU sample closest to the current frame
zed.getSensorsData(sensors_data, TIME_REFERENCE::IMAGE);
SensorsData::IMUData imu = sensors_data.imu;
// imu.linear_acceleration : accelerometer, in m/s²
// imu.angular_velocity : gyroscope, in deg/s
// imu.pose : orientation (IMU 6-DoF fusion)
}
}

Like every other module, those that rely on sensor data can be run at playback time even if they were not enabled during the recording session. You can, for example, call enablePositionalTracking() while reading an SVO2 file and get a trajectory from a recording made without positional tracking.

Cameras without an IMU record no motion data, so nothing can be recovered from their SVO files. Check the sensor availability table for your camera model.

Converting and Exporting SVO Files

Converting SVO to SVO2 or to Another Compression Mode

The ZED SVO Editor command-line tool, installed with the ZED SDK, re-encodes an SVO file with the -cpr option. The output file is written in the SVO2 format, so the same command converts a legacy SVO file to SVO2 and changes its compression mode:

# Convert a legacy SVO file to SVO2, re-encoded in H.265
ZED_SVO_Editor -cpr input.svo -m 4 output.svo2

The -m option selects the compression mode of the output file:

-mCompression mode
0LOSSLESS (PNG)
1LOSSLESS (ZSTD)
2JPEG (lossy, CPU based, not available in the ZED API)
3H.264 (AVCHD)
4H.265 (HEVC)
5H.264 LOSSLESS
6H.265 LOSSLESS

Use ZED_SVO_Editor -inf <file> to check the format version (SVO v 2 for SVO2) and the compression mode of a file, and ZED_SVO_Editor --help for its other options, such as cutting, merging, repairing, encrypting and decrypting files.

Converting a legacy SVO file to SVO2 does not add the high-frequency sensor data that the file never contained: its IMU samples stay at the image frame rate. Features that require high-frequency IMU data, such as positional tracking GEN_2 with IMU fusion, still need a file recorded in SVO2.

Exporting to Video Files or Image Sequences

An SVO file can only be read with the ZED SDK. To use the recorded images with other tools, export them with the SVO Export sample (C++, Python, C#). It plays back the file and writes, depending on the export mode:

ModeOutput
0AVI video, left and right images side by side
1AVI video, left image and depth view side by side
2PNG image sequence, left and right images
3PNG image sequence, left image and depth view
4PNG image sequence, left image and 16-bit depth map in millimeters
# C++ sample: <input SVO> <output AVI file or folder> <export mode>
ZED_SVO_Export input.svo2 output.avi 0 # left + right AVI
ZED_SVO_Export input.svo2 output_folder/ 4 # left + 16-bit depth PNG sequence

The sample relies on OpenCV to write the files. Modify it to export other views, other file formats or other data, for example to write an MP4 file, which may require OpenCV built with FFmpeg support.

Other export options:

  • Sensor data: read the IMU and other sensor data with getSensorsData(), as shown in Extracting IMU Data from an SVO2 File, and write it in the format you need. The C++ version of the Sensor Data export sample is a starting point: it writes the sensor data of the SVO file passed as argument to Sensors.json. It only stops after 4000 IMU samples, so with a shorter recording, make it stop when grab() returns END_OF_SVOFILE_REACHED.
  • MCAP: ZED_SVO_Editor -export-to-mcap input.svo2 exports the images, the IMU data and the custom data of an SVO2 file to an MCAP file that can be opened in Foxglove. The tool asks for confirmation and for the output file name, input.mcap by default.
  • ROS 2 bag: replay the SVO file with the ZED ROS 2 Wrapper and record the published topics, as described in Record and Replay Camera Data with ROS 2.

Code Example

Check out the SVO Recording, SVO Playback and SVO Export samples on GitHub.