Tutorial - Split Process

This tutorial shows how to decouple image and depth acquisition using the read() and grab() functions, so images can be retrieved at full frame rate while depth and point cloud measurements are only computed every few frames. The program captures 150 frames, computing depth every 6th frame, and prints the resulting image and depth throughput (FPS). We assume that you have followed the Depth Sensing tutorial before.

Getting Started

First, download the latest version of the ZED SDK.
Download the Split Process sample code in C++.

Code Overview

Open the camera

As in previous tutorials, we create, configure and open the ZED, enabling NEURAL depth mode.

C++

1 // Create a ZED camera object
2 Camera zed;
3 
4 // Set configuration parameters
5 InitParameters init_parameters;
6 init_parameters.depth_mode = DEPTH_MODE::NEURAL;     // Use NEURAL depth mode
7 init_parameters.coordinate_units = UNIT::MILLIMETER; // Use millimeter units (for depth measurements)
8 
9 // Open the camera
10 auto returned_state = zed.open(init_parameters);
11 if (returned_state > ERROR_CODE::SUCCESS) {
12     cout << "Error " << returned_state << ", exit program." << endl;
13     return EXIT_FAILURE;
14 }

Capture images with `read()`, depth with `grab()`

In previous tutorials, grab() was called on every iteration to get both the image and the depth. Here, we call zed.read() on every loop to retrieve the left image as fast as possible, and only call zed.grab() (which recomputes depth) every 6 frames — reducing the computational cost when full-rate depth isn’t needed.

C++

1 // Capture 150 images then stop
2 int frame_count = 0;
3 int depth_count = 0;
4 sl::Mat image, depth, point_cloud;
5 
6 const sl::Timestamp start_ts = sl::getCurrentTimeStamp();
7 const int depth_every_n_frames = 6;
8 
9 while (frame_count < 150) {
10     // A new image is available if read() returns ERROR_CODE::SUCCESS
11     if (zed.read() <= ERROR_CODE::SUCCESS) {
12         // Retrieve left image
13         zed.retrieveImage(image, VIEW::LEFT);
14         frame_count++;
15     }
16 
17     // Measurements are available if grab() returns ERROR_CODE::SUCCESS or a WARNING (an error_code lower than ERROR_CODE::SUCCESS)
18     if (((frame_count % depth_every_n_frames) == 0) && (zed.grab() <= ERROR_CODE::SUCCESS)) {
19         // Retrieve depth map. Depth is aligned on the left image
20         zed.retrieveMeasure(depth, MEASURE::DEPTH);
21         // Retrieve colored point cloud. Point cloud is aligned on the left image.
22         zed.retrieveMeasure(point_cloud, MEASURE::XYZRGBA);
23 
24         // Get and print distance value in mm at the center of the image
25         // We measure the distance camera - object using Euclidean distance
26         const int x = point_cloud.getWidth() / 2;
27         const int y = point_cloud.getHeight() / 2;
28         sl::float4 point_cloud_value;
29         point_cloud.getValue(x, y, &point_cloud_value);
30 
31         if (std::isfinite(point_cloud_value.z)) // convert to float3 to use norm(), the 4th component is used to store the color
32             cout << "Distance to Camera at {" << x << ";" << y << "}: " << sl::float3(point_cloud_value).norm() << "mm" << endl;
33         else
34             cout << "The Distance can not be computed at {" << x << ";" << y << "}" << endl;
35         depth_count++;
36     }
37 }

read() grabs a new image without recomputing depth, which is significantly cheaper on GPU than grab(). Use this pattern when your application needs images at the sensor’s full frame rate but can tolerate a lower depth-update rate.

For more information on read() and grab(), see Using the Video API.

Report throughput and close the camera

Once 150 frames have been captured, we compute and print the resulting image and depth FPS, then close the camera.

C++

1 auto diff = (sl::getCurrentTimeStamp() - start_ts).getSeconds();
2 // Print the FPS
3 if (diff > 0)
4     std::cout << "Image: " << (frame_count / diff) << "FPS / Depth: " << (depth_count / diff) << "FPS" << std::endl;
5 
6 // Close the camera
7 zed.close();