Tutorial - Using Depth Perception

This tutorial shows how to retrieve the depth from a stereo image and point cloud and print the distance of a given pixel in the terminal. The program will loop until 50 frames are grabbed. We assume that you have followed the previous tutorials: Hello ZED and Image Capture.

Getting Started

First, download the latest version of the ZED SDK.
Download the Depth Sensing sample code in C++, Python or C#.

Code Overview

Open the camera

As in previous tutorials, we create, configure and open the ZED. We set the 3D camera in HD720 mode at 60fps and enable depth in PERFORMANCE mode. For more information on this parameter, see Depth Modes.

1 // Create a ZED camera
2 Camera zed;
3 // Create configuration parameters
4 InitParameters init_params;
5 init_params.sdk_verbose = true; // Enable verbose logging
6 init_params.depth_mode = DEPTH_MODE::PERFORMANCE; // Set the depth mode to performance (fastest)
7 init_params.coordinate_units = UNIT::MILLIMETER; // Use millimeter units
8 // Open the camera
9 ERROR_CODE err = zed.open(init_params);
10 if (err != ERROR_CODE::SUCCESS) {
11     cout << "Error " << err << ", exit program.\n"; // Display the error
12     return -1;
13 }

By default, depth perception is in NEURAL mode. If you’re using this mode, it is not necessary to set the depth mode in InitParameters.

Capture Image and Depth

Now that the ZED is opened, we can capture images and depth. Here we loop until we have successfully captured 50 images. Retrieving the depth map is as simple as retrieving an image:

We create a Mat to store the depth map.
We call retrieveMeasure() to get the depth map.

1 // Capture 50 images and depth, then stop
2 int i = 0;
3 sl::Mat image, depth, point_cloud;
4 while (i < 50) {
5     // Grab an image
6     if (zed.grab(runtime_parameters) == ERROR_CODE::SUCCESS) {
7         // A new image is available if grab() returns ERROR_CODE::SUCCESS
8         zed.retrieveImage(image, VIEW::LEFT); // Get the left image
9         zed.retrieveMeasure(depth, MEASURE::DEPTH); // Retrieve depth matrix. Depth is aligned on the left RGB image.
10         zed.retrieveMeasure(point_cloud, MEASURE::XYZRGBA); // Retrieve colored point cloud
11         i++;
12     }
13 }

For more information on depth and point cloud parameters, read Using the Depth API.

Measure Distance in Point Cloud

Now that we have retrieved the point cloud, we can extract the depth at a specific pixel. In the example, we extract the distance of the point at the center of the image (width/2, height/2).

1 // Get and print distance value in mm at the center of the image
2 // We measure the distance camera - object using Euclidean distance
3 int x = image.getWidth() / 2;
4 int y = image.getHeight() / 2;
5 sl::float4 point_cloud_value;
6 point_cloud.getValue(x, y, &point_cloud_value);
7 float distance = sqrt(point_cloud_value.x*point_cloud_value.x + point_cloud_value.y*point_cloud_value.y + point_cloud_value.z*point_cloud_value.z);
8 printf("Distance to Camera at (%d, %d): %f mm\n", x, y, distance);

We can also use directly the depth map instead of the point cloud to extract the depth value at a specific pixel.

1 int x = image.getWidth() / 2;
2 int y = image.getHeight() / 2;
3 float depth_value;
4 depth.getValue(x,y, depth_value);
5 printf("Depth to Camera at (%d, %d): %f mm\n", x, y, depth_value);

Close the camera

Once 50 frames have been grabbed, we close the camera.

1 // Close the camera
2 zed.close();

Advanced Example

To learn how to retrieve and display the live point cloud from the camera and adjust depth confidence filters, check the Point Cloud Viewer sample code.