Tutorial - Using Human Body Tracking

This tutorial shows how to use your ZED 3D camera to detect and track human bodies using the ZED Body Tracking module.

Getting Started

First, download the latest version of the ZED SDK.
Download the Body Tracking sample code in C++, Python or C#.

Code Overview

Open the camera

In this tutorial, we will use the Body Tracking AI module of the ZED SDK. As in previous tutorials, we create, configure and open the camera.

1 // Create ZED objects
2 Camera zed;
3 InitParameters initParameters;
4 initParameters.camera_resolution = RESOLUTION::HD720;
5 initParameters.depth_mode = DEPTH_MODE::ULTRA;
6 
7 // Open the camera
8 ERROR_CODE zed_error = zed.open(initParameters);
9 if (zed_error != ERROR_CODE::SUCCESS) {
10 	std::cout << "Error " << zed_error << ", exit program.\n";
11 	return 1; // Quit if an error occurred
12 }

Enable 3D Object detection

Before enabling body tracking, we specify the BodyTrackingParameters of the module.

Body tracking needs positional tracking to be able to track the bodies in the world reference frame.

1 // Define the Objects detection module parameters
2 BodyTrackingParameters detection_parameters;
3 // Different models can be chosen, optimizing the runtime or the accuracy
4 detection_parameters.detection_model = BODY_TRACKING_MODEL::HUMAN_BODY_FAST;
5 // Run detection for every Camera grab
6 detection_parameters.image_sync = true;
7 // Enable tracking to detect objects across time and space
8 detection_parameters.enable_tracking = true;
9 // Optimize the person joints position, requires more computations
10 detection_parameters.enable_body_fitting = true;
11 
12 // If you want to have object tracking you need to enable positional tracking first
13 if (detection_parameters.enable_tracking)
14     zed.enablePositionalTracking();

Then we can start the module, it will load the model. This operation can take a few seconds. The first time the module is used, the model will be optimized for the hardware and will take more time. This operation is done only once.

1 cout << "Body Tracking: Loading Module..." << endl;
2 returned_state = zed.enableObjectDetection(detection_parameters);
3 if (returned_state != ERROR_CODE::SUCCESS) {
4     cout << "Error " << returned_state << ", exit program.\n";
5     zed.close();
6     return EXIT_FAILURE;
7 }

Capture Data

The object confidence threshold can be adjusted at runtime to select only the relevant skeletons depending on the scene complexity. Since the parameters have been set to image_sync, for each grab call, the image will be fed into the AI module and will output the detections for each frame.

1 // Set runtime parameter confidence to 40
2 ObjectDetectionRuntimeParameters detection_parameters_rt;
3 detection_parameters_rt.detection_confidence_threshold = 40;
4 
5 Objects objects;
6 
7 // Grab new frames and detect objects
8 while (zed.grab() == ERROR_CODE::SUCCESS) {
9 	err = zed.retrieveObjects(objects, detection_parameters_runtime);
10 	if (objects.is_new) {
11 		// Count the number of objects detected
12 		cout << objects.object_list.size() << " Person(s) detected\n";
13 		if (!objects.object_list.empty()) {
14 			auto first_object = objects.object_list.front();
15 			// Display the 3D keypoint coordinates of the first detected person    
16 			cout << " Keypoints 3D \n";
17 			for (int i = 0; i < first_object.keypoint.size(); i++) {
18 				auto &kp = first_object.keypoint[i];
19 				cout << "    " << kp.x << ", " << kp.y << ", " << kp.z << "\n";
20 			}
21 		}
22 	}
23 }

Disable modules and exit

Once the program is over the modules can be disabled and the camera closed. This step is optional since the zed.close() will take care of disabling all the modules. This function is also called automatically by the destructor if necessary.

1 // Disable body tracking and close the camera
2 zed.disableBodyTracking();
3 zed.close();

And this is it!