Tutorial - Retrieve Tensor

This tutorial shows how to use Camera::retrieveTensor() to get a pre-processed image tensor directly from the ZED camera — resized, normalized and laid out in NCHW format — ready to feed into a deep learning inference pipeline without writing custom pre-processing code.

Getting Started

First, download the latest version of the ZED SDK.
Download the Retrieve Tensor sample code in C++.

Code Overview

Open the camera

As in previous tutorials, we create, configure and open the ZED. Depth is not needed for this tutorial, so we set DEPTH_MODE::NONE.

C++

1 // Create a ZED camera object
2 sl::Camera zed;
3 
4 // Set configuration parameters
5 sl::InitParameters init_parameters;
6 init_parameters.camera_resolution = sl::RESOLUTION::HD720;
7 init_parameters.camera_fps = 30;
8 init_parameters.depth_mode = sl::DEPTH_MODE::NONE;
9 
10 // Open the camera
11 sl::ERROR_CODE err = zed.open(init_parameters);
12 if (err > sl::ERROR_CODE::SUCCESS) {
13     std::cout << "Error " << err << ", exit program." << std::endl;
14     return EXIT_FAILURE;
15 }

Configure the tensor parameters

TensorParameters describes the output format expected by your inference pipeline: target resolution, batch size, memory layout (NCHW/NHWC), pixel type, color format, and the normalization to apply (scale, mean, std). Here we target a 501x501 RGB float tensor on GPU memory, scaled to [0, 1] with no additional mean/std normalization.

C++

1 // Prepare Tensor buffer
2 sl::Tensor tensor;
3 
4 // Tensor parameters - Configure the output format for deep learning inference
5 sl::TensorParameters tensor_params;
6 tensor_params.target_size = sl::Resolution(501, 501);
7 tensor_params.batch_size = 1;
8 tensor_params.layout = sl::TensorParameters::LAYOUT::NCHW;
9 tensor_params.pixel_type = sl::TensorParameters::PIXEL_TYPE::FLOAT;
10 tensor_params.color_format = sl::TensorParameters::COLOR_FORMAT::RGB;
11 tensor_params.memory_type = sl::MEM::GPU;
12 tensor_params.scale = sl::float3(1.0f / 255.0f, 1.0f / 255.0f, 1.0f / 255.0f);
13 tensor_params.stretch = true;
14 // The default normalization uses ImageNet statistics (mean={0.485, 0.456, 0.406}, std={0.229, 0.224, 0.225});
15 // here we disable normalization instead by using a neutral mean/std
16 tensor_params.mean = sl::float3(0.0f, 0.0f, 0.0f);
17 tensor_params.std = sl::float3(1.0f, 1.0f, 1.0f);

For the full list of available options, see the TensorParameters API documentation.

Retrieve the tensor

On each grab(), retrieveTensor() fills the Tensor object with the pre-processed image data, ready for inference. The sample captures 50 frames and prints the tensor dimensions every 10 frames.

C++

1 // Main loop
2 int frame_count = 0;
3 while (frame_count < 50) {
4     if (zed.grab() <= sl::ERROR_CODE::SUCCESS) {
5         // Retrieve pre-processed tensor for inference
6         sl::ERROR_CODE res = zed.retrieveTensor(tensor, tensor_params);
7 
8         if (res <= sl::ERROR_CODE::SUCCESS) {
9             if (frame_count % 10 == 0) {
10                 std::vector<size_t> dims = tensor.getDims();
11                 std::cout << "Frame " << frame_count << ": Dims: [" << dims[0] << ", " << dims[1] << ", " << dims[2] << ", " << dims[3]
12                           << "]" << std::endl;
13             }
14         } else {
15             std::cout << "Error retrieving tensor: " << res << std::endl;
16         }
17         frame_count++;
18     }
19 }

The full sample also includes a helper function that downloads the tensor from GPU to CPU and reverses the normalization to save it back as a PNG image, which is a convenient way to visually check your pre-processing pipeline. See the source code for the full implementation.

Close the camera

C++

1 zed.close();