Software IntegrationOpenCV

How to Use OpenCV with ZED in Python

Open in ClaudeOpen in ChatGPT

Introduction

In this tutorial, you will learn how to capture and display color and depth images using OpenCV and the ZED SDK in Python.

image

Code Example

The sample code is available on GitHub. Make sure the ZED Python API is installed before launching the sample.

Sharing image data between ZED SDK and OpenCV Python

In Python, OpenCV stores images in NumPy arrays. Since the ZED SDK uses its own sl.Mat class to store image data, we provide a function get_data() to convert the sl.Mat matrix into a NumPy array.

Python
1# Create an RGBA sl.Mat object
2image_zed = sl.Mat(zed.get_camera_information().camera_resolution.width, zed.get_camera_information().camera_resolution.height, sl.MAT_TYPE.U8_C4)
3# Retrieve data in a numpy array with get_data()
4image_ocv = image_zed.get_data()

Capturing Video

To capture video, use grab() and retrieve_image(). Then use get_data() to retrieve the sl.Mat data into a NumPy array. Display the video using cv2.imshow().

Python
1if zed.grab() == sl.ERROR_CODE.SUCCESS :
2    # Retrieve the left image in sl.Mat
3    zed.retrieve_image(image_zed, sl.VIEW.LEFT)
4    # Use get_data() to get the numpy array
5    image_ocv = image_zed.get_data()
6    # Display the left image from the numpy array
7    cv2.imshow("Image", image_ocv)

Capturing Depth

A depth map is a 1-channel matrix with 32-bit float values for each pixel. Each value expresses the distance of a pixel in the scene. The depth map can be retrieved using retrieve_measure() and loaded with get_data() into a NumPy array. Please refer to the Depth API for more information.

Python
1# Create a sl.Mat with float type (32-bit)
2depth_zed = sl.Mat(zed.get_camera_information().camera_resolution.width, zed.get_camera_information().camera_resolution.height, sl.MAT_TYPE.F32_C1)
3
4if zed.grab() == sl.ERROR_CODE.SUCCESS :
5    # Retrieve depth data (32-bit)
6    zed.retrieve_measure(depth_zed, sl.MEASURE.DEPTH)
7    # Load depth data into a numpy array
8    depth_ocv = depth_zed.get_data()
9    # Print the depth value at the center of the image
10    print(depth_ocv[int(len(depth_ocv)/2)][int(len(depth_ocv[0])/2)])

Displaying Depth

A NumPy array with 32-bit float values can’t be displayed with cv2.imshow. To display the depth map, we need to normalize the depth values between 0 and 255 (8-bit) and create a black-and-white representation. Do not use this representation for other purposes than displaying the image.

Python
1# Create an RGBA sl.Mat object
2image_depth_zed = sl.Mat(zed.get_camera_information().camera_resolution.width, sl.get_camera_information().camera_resolution.height, sl.MAT_TYPE.U8_C4)
3
4if zed.grab() == sl.ERROR_CODE.SUCCESS :
5    # Retrieve the normalized depth image
6    zed.retrieve_image(image_depth_zed, sl.VIEW.DEPTH)
7    # Use get_data() to get the numpy array
8    image_depth_ocv = image_depth_zed.get_data()
9    # Display the depth view from the numpy array
10    cv2.imshow("Image", image_depth_ocv)

UVC Capture

You can also use the ZED as a standard UVC camera in OpenCV to capture raw stereo video using the code snippet below. To get rectified images and calibration with OpenCV, use the native (Python) capture sample available on GitHub.

Python
1import cv2
2import numpy
3
4# Open the ZED camera
5cap = cv2.VideoCapture(0)
6if cap.isOpened() == 0:
7    exit(-1)
8
9# Set the video resolution to HD720 (2560*720)
10cap.set(cv2.CAP_PROP_FRAME_WIDTH, 2560)
11cap.set(cv2.CAP_PROP_FRAME_HEIGHT, 720)
12
13while True :
14    # Get a new frame from camera
15    retval, frame = cap.read()
16    # Extract left and right images from side-by-side
17    left_right_image = numpy.split(frame, 2, axis=1)
18    # Display images
19    cv2.imshow("frame", frame)
20    cv2.imshow("right", left_right_image[0])
21    cv2.imshow("left", left_right_image[1])
22    if cv2.waitKey(30) >= 0 :
23        break
24
25exit(0)