Body Tracking with ROS 2

The ROS 2 wrapper offers full support for the Body Tracking module of the ZED SDK.

The Body Tracking module can be configured to use one of the available detection models:

MODEL	Description
`HUMAN_BODY_FAST`	Keypoints based, specific to human skeleton, real time performance even on NVIDIA® Jetson™ or low-end GPU cards
`HUMAN_BODY_MEDIUM`	Keypoints based, specific to human skeleton, compromise between accuracy and speed
`HUMAN_BODY_ACCURATE`	Keypoints based, specific to human skeleton, state of the art accuracy, requires powerful GPU

The Body Tracking module can be configured to use one of the available body formats:

FORMAT	Description
`BODY_18`	18 keypoints model. Basic Body model
`BODY_34`	34 keypoints model (default). Body model, requires body fitting enabled
`BODY_38`	38 keypoints model. Body model, including feet, simplified face, and hands

Starting from ZED SDK 5.5 the module can also be configured to select the network generation with the body_tracking.model_gen parameter:

GENERATION	Description
`GEN_1`	The network used up to ZED SDK 5.4
`GEN_2`	Default. Bottom-up network: more robust in crowded scenes and to unusual body orientations, detects people further from the camera, and is up to 18% faster and 14% more accurate than `GEN_1`. It always runs in FP16, so `body_tracking.allow_reduced_precision_inference` has no effect on it

Only BODY_18 and BODY_34 have a GEN_2 network: with BODY_38 the ZED SDK falls back to GEN_1.

Since v5.5.0 of the ROS 2 Wrapper the default body format is BODY_34 instead of BODY_38, and body_tracking.enable_body_fitting defaults to true, so that the default configuration actually runs GEN_2. Skeletons published on ~/body_trk/skeletons now carry 34 keypoints instead of 38: set body_format back to BODY_38 to keep the previous topology, at the cost of running GEN_1.

The result of the detection is published using a new custom message of type zed_msgs/ObjectsStamped defined in the package zed_msgs.

Enable Body Tracking

Body Tracking can be started automatically when the ZED Wrapper node starts by setting the parameter body_tracking.bt_enabled to true in the file common_stereo.yaml.

It is also possible to start the Body Tracking processing manually by calling the service ~/enable_body_trk with the parameter True.

In both cases, the Body Tracking processing can be stopped by calling the service ~/enable_body_trk with the parameter False.

See the services documentation for more info.

Body Tracking results in RViz 2

To visualize the results of the Body Tracking processing in RViz 2, the new ZedOdDisplay plugin is required. The plugin is available in the zed-ros2-examples GitHub repository and can be installed following the online instructions.

The source code of the plugin is a valid example of how to process the data of the topics of type zed_msgs/ObjectsStamped.

Parameters:

Topic: Selects the body tracking topic to visualize from the list of available images in the combo box.
Depth: The depth of the incoming message queue.
History policy: Set the QoS history policy. Keep Last is suggested for performance and compatibility.
Reliability Policy: Set the QoS reliability policy. Best Effort is suggested for performance and compatibility.
Durability Policy: Set the QoS durability policy. Volatile is suggested for compatibility.
Transparency: the transparency level of the structures composing the detected bodies.
Show skeleton: enable/disable the visualization of the skeleton of the detected persons.
Show Labels: enable/disable the visualization of the label.
Show Bounding Boxes: enable/disable the visualization of the bounding boxes of the detected bodies.
Link Size: the size of the bounding boxes’ corner lines and skeleton link lines.
Joint Radius: the radius of the spheres placed on the corners of the bounding boxes and on the skeleton joint points.
Label Scale: the scale of the label of the bodies.