> For clean Markdown of any page, append .md to the page URL. > For a complete documentation index, see https://docs.stereolabs.com/llms.txt. > For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.stereolabs.com/_mcp/server. # Adding Object Detection in ROS > **Warning** > > **The ZED ROS Wrapper packages are no longer maintained** because ROS 1 reached [end-of-life (EOL) on 2025-05-31](https://www.ros.org/blog/noetic-eol/) with the final Noetic release. > > We recommend upgrading to [ROS 2](/docs/integrations/ros-2) to take advantage of the latest ZED SDK features for robotics applications. ## Object Detection with RViz The ROS wrapper offers full support for the Object Detection module of the ZED SDK. The Object Detection module is not available with the **ZED** camera. The Object Detection module can be [configured](/docs/integrations/ros/zed-node/#zed-parameters) to use one of the following detection models: * **MULTI\_CLASS\_BOX\_FAST**: Any objects, bounding box based. * **MULTI\_CLASS\_BOX\_MEDIUM**: Any objects, bounding box based, compromise between accuracy and speed. * **MULTI\_CLASS\_BOX\_ACCURATE**: Any objects, bounding box based, more accurate but slower than the base model. * **PERSON\_HEAD\_BOX\_FAST**: Bounding box detector specialized in person heads particularly well suited for crowded environments. The person localization is also improved. * **PERSON\_HEAD\_BOX\_ACCURATE**: Bounding box detector specialized in person heads, particularly well suited for crowded environments. The person localization is also improved, more accurate but slower than the base model. The result of the detection is published using a new custom message of type [zed\_interfaces/ObjectsStamped](#detected-objects-message) defined in the package [zed\_interfaces](https://github.com/stereolabs/zed-ros-interfaces). ## Enable Object Detection Object detection can be started automatically when the ZED Wrapper node starts by setting the parameter `object_detection/od_enabled` to `true` in the file `params/common.yaml`. It is also possible to start the Object Detection processing manually by calling the service `enable_object_detection` (type `std_srvs/SetBool`) with `true` as the parameter. In both cases, the Object Detection processing can be stopped by calling the service `enable_object_detection` with `false` as the parameter. See the [services](/docs/integrations/ros/zed-node/#services) documentation for more info. ## Object Detection results in RViz To visualize the results of the Object Detection processing in RViz, the `ZedOdDisplay` plugin is required. The plugin is available in the [`zed-ros-examples`](https://github.com/stereolabs/zed-ros-examples) GitHub repository and can be installed following the [online instructions](https://github.com/stereolabs/zed-ros-examples#build-the-program). > **Note** > > [The source code of the plugin](https://github.com/stereolabs/zed-ros-examples/tree/master/rviz-plugin-zed-od) is a valid example of how to process the data of the topics of type [zed\_interfaces/ObjectsStamped](#detected-objects-message). ![image](/_fern-img/cf7c55ab78a316535a435cabbbc0fa64e96095249d563177f694af466733e88b.webp) Parameters: * `Topic`: Selects the object detection topic to visualize from the list of available topics in the combo box. * `Transparency`: the transparency level of the structures composing the detected objects. * `Show skeleton`: enable/disable the visualization of the skeleton of the detected persons (if available). * `Show Labels`: enable/disable the visualization of the object label. * `Show Bounding Boxes`: enable/disable the visualization of the bounding boxes of the detected objects. * `Link Size`: the size of the bounding boxes' corner lines and skeleton link lines. * `Joint Radius`: the radius of the spheres placed on the corners of the bounding boxes and on the skeleton joint points. * `Label Scale`: the scale of the label of the object. ![image](https://cdn.sanity.io/images/s18ewfw4/production/f6ed27bc3c83d0b4ad99c26bc1bed49bdeb30175-1877x1049.webp) ## Detected Objects message The `zed_interfaces/ObjectsStamped` message is defined as: ```bash # Standard Header std_msgs/Header header # Array of detected objects zed_interfaces/Object[] objects ``` where `zed_interfaces/Object` is defined as: ```bash # Object label string label # Object label ID int16 label_id # Object instance ID int16 instance_id # Object sub-class label string sublabel # Object confidence level (1-99) float32 confidence # Object centroid position float32[3] position # Position covariance float32[6] position_covariance # Object velocity float32[3] velocity # Tracking available bool tracking_available # Tracking state # 0 -> OFF (object not valid) # 1 -> OK # 2 -> SEARCHING (occlusion occurred, trajectory is estimated) # 3 -> TERMINATE (last searching state, the track will be deleted in the next frame) int8 tracking_state # Action state # 0 -> IDLE # 1 -> MOVING int8 action_state # 2D Bounding box projected to Camera image zed_interfaces/BoundingBox2Di bounding_box_2d # 3D Bounding box in world frame zed_interfaces/BoundingBox3D bounding_box_3d # 3D dimensions (width, height, length) float32[3] dimensions_3d # Is skeleton available? bool skeleton_available # 2D Bounding box projected to Camera image of the person head zed_interfaces/BoundingBox2Df head_bounding_box_2d # 3D Bounding box in world frame of the person head zed_interfaces/BoundingBox3D head_bounding_box_3d # 3D position of the centroid of the person head float32[3] head_position # 2D Person skeleton projected to Camera image zed_interfaces/Skeleton2D skeleton_2d # 3D Person skeleton in world frame zed_interfaces/Skeleton3D skeleton_3d ``` And all the sub-messages are defined as follows: `zed_interfaces/BoundingBox2Df`: ```bash # 0 ------- 1 # | | # | | # | | # 3 ------- 2 zed_interfaces/Keypoint2Df[4] corners ``` `zed_interfaces/BoundingBox2Di`: ```bash # 0 ------- 1 # | | # | | # | | # 3 ------- 2 zed_interfaces/Keypoint2Di[4] corners ``` `zed_interfaces/BoundingBox3D`: ```bash # 1 ------- 2 # /. /| # 0 ------- 3 | # | . | | # | 5.......| 6 # |. |/ # 4 ------- 7 zed_interfaces/Keypoint3D[8] corners ``` `zed_interfaces/Keypoint2Df`: ```bash float32[2] kp ``` `zed_interfaces/Keypoint2Di`: ```bash uint32[2] kp ``` `zed_interfaces/Keypoint3D`: ```bash float32[3] kp ``` `zed_interfaces/Skeleton2D`: ```bash # Skeleton joints indices # 16-14 15-17 # \ / # 0 # | # 2------1------5 # | | | | # | | | | # 3 | | 6 # | | | | # | | | | # 4 8 11 7 # | | # | | # | | # 9 12 # | | # | | # | | # 10 13 zed_interfaces/Keypoint2Df[18] keypoints ``` `zed_interfaces/Skeleton3D`: ```bash # Skeleton joints indices # 16-14 15-17 # \ / # 0 # | # 2------1------5 # | | | | # | | | | # 3 | | 6 # | | | | # | | | | # 4 8 11 7 # | | # | | # | | # 9 12 # | | # | | # | | # 10 13 zed_interfaces/Keypoint3D[18] keypoints ``` ![image](/_fern-img/05782b1f9ed92f7b61e5dbbfd97914610d9803c14c354c33b8e659018045080d.webp) ## Object list subscribing in C++ In this tutorial, you will learn how to write a simple C++ node that subscribes to messages of type `zed_interfaces/ObjectsStamped`. This lets you retrieve the list of detected objects published by the ZED node for each camera frame. ### Introduction Use this command to connect a ZED 2 camera to the ROS network: ```bash roslaunch zed_wrapper zed2.launch ``` or the corresponding launch file for your camera model (`zedm.launch`, `zed2i.launch`, `zedx.launch` or `zedxm.launch`), for example with a ZED 2i: ```bash roslaunch zed_wrapper zed2i.launch ``` The ZED node will start publishing object detection data to the network only if there is another node subscribing to the relevant topic and if the Object Detection module has been started. > **Note** > > The Object Detection module in the ZED wrapper can start automatically only if the parameter [`object_detection/od_enabled`](/docs/integrations/ros/zed-node/#zed-parameters) in `params/common.yaml` is set to `true` (default `false`). To start the module manually, you can call the service [`enable_object_detection`](/docs/integrations/ros/zed-node/#services) with `true` as the parameter. ### Running the tutorial If you properly followed the [ROS Installation Guide](/docs/integrations/ros/#installation), the executable of this tutorial has been compiled and you can run the subscriber node using these commands: ```bash rosrun zed_obj_det_sub_tutorial zed_obj_det_sub objects:=/zed2/zed_node/obj_det/objects ``` Replace `zed2` with the name of your camera model (for example `zed2i`) if you are not using a ZED 2. If the ZED node is running and a camera is connected or you have loaded an SVO file, you will receive the following stream of messages confirming that you have correctly subscribed to the ZED object detection topic: ```bash [ INFO] [1614264700.322685566]: ***** New object list ***** [ INFO] [1614264700.322717010]: Person [0] - Pos. [0.808921,-0.364787,-0.215571] [m]- Conf. 78.0129 - Tracking state: 1 [ INFO] [1614264700.370680525]: ***** New object list ***** [ INFO] [1614264700.370712147]: Person [0] - Pos. [0.809903,-0.365017,-0.215609] [m]- Conf. 80.6333 - Tracking state: 1 [ INFO] [1614264700.418698143]: ***** New object list ***** [ INFO] [1614264700.418729581]: Person [0] - Pos. [0.810399,-0.365178,-0.215623] [m]- Conf. 82.0206 - Tracking state: 1 [ INFO] [1614264700.426152141]: ***** New object list ***** [ INFO] [1614264700.426188225]: Person [0] - Pos. [0.810885,-0.365334,-0.215639] [m]- Conf. 80.0525 - Tracking state: 1 [ INFO] [1614264700.466679371]: ***** New object list ***** [ INFO] [1614264700.466709900]: Person [0] - Pos. [0.811356,-0.365492,-0.215652] [m]- Conf. 80.8249 - Tracking state: 1 [ INFO] [1614264700.514674426]: ***** New object list ***** [ INFO] [1614264700.514705119]: Person [0] - Pos. [0.811856,-0.365647,-0.215659] [m]- Conf. 83.1162 - Tracking state: 1 [ INFO] [1614264700.562679710]: ***** New object list ***** [ INFO] [1614264700.562711759]: Person [0] - Pos. [0.812835,-0.366073,-0.215665] [m]- Conf. 81.5075 - Tracking state: 1 [ INFO] [1614264700.610687663]: ***** New object list ***** [ INFO] [1614264700.610717479]: Person [0] - Pos. [0.813346,-0.366275,-0.215661] [m]- Conf. 81.6733 - Tracking state: 1 [ INFO] [1614264700.626001568]: ***** New object list ***** [ INFO] [1614264700.626036093]: Person [0] - Pos. [0.813875,-0.366408,-0.215656] [m]- Conf. 81.7185 - Tracking state: 1 [...] ``` Where the "Tracking state" values can be decoded as: * 0 -> OFF (object not valid) * 1 -> OK * 2 -> SEARCHING (trajectory is estimated) * 3 -> TERMINATE (the track will be deleted in the next frame) ### The code The source code of the subscriber node `zed_obj_det_sub_tutorial.cpp`: **`C++`** ```cpp C++ #include #include #include /** * Subscriber callback */ void objectListCallback(const zed_interfaces::ObjectsStamped::ConstPtr& msg) { ROS_INFO("***** New object list *****"); for (int i = 0; i < msg->objects.size(); i++) { if (msg->objects[i].label_id == -1) continue; ROS_INFO_STREAM(msg->objects[i].label << " [" << msg->objects[i].label_id << "] - Pos. [" << msg->objects[i].position[0] << "," << msg->objects[i].position[1] << "," << msg->objects[i].position[2] << "] [m]" << "- Conf. " << msg->objects[i].confidence << " - Tracking state: " << static_cast(msg->objects[i].tracking_state)); } } /** * Node main function */ int main(int argc, char** argv) { ros::init(argc, argv, "zed_obj_det_sub_tutorial"); ros::NodeHandle n; // Subscriber ros::Subscriber subObjList = n.subscribe("objects", 1, objectListCallback); ros::spin(); return 0; } ``` ### The code explained The following is a brief explanation of the source code above: **`C++`** ```cpp C++ void objectListCallback(const zed_interfaces::ObjectsStamped::ConstPtr& msg) { ROS_INFO( "***** New object list *****"); for(int i=0; iobjects.size();i++) { if(msg->objects[i].label_id == -1) continue; ROS_INFO_STREAM( msg->objects[i].label << " [" << msg->objects[i].label_id << "] - Pos. [" << msg->objects[i].position[0] << "," << msg->objects[i].position[1] << "," << msg->objects[i].position[2] << "] [m]" << "- Conf. " << msg->objects[i].confidence << " - Tracking state: " << static_cast(msg->objects[i].tracking_state) ); } } ``` This callback is executed when the subscriber node receives a message of type `zed_interfaces/ObjectsStamped` that matches the subscribed topic. The parameter of the callback is a `boost::shared_ptr` to the received message. This means you don't have to worry about memory management. The callback code is very simple and demonstrates how to access the fields in a message; in this case, the object list and for each object its `label` and `label_id`, the `position` and the `tracking_state`. The `main` function is very standard and is explained in detail in the ["Talker/Listener" ROS tutorial](http://wiki.ros.org/ROS/Tutorials/WritingPublisherSubscriber%28c%2B%2B%29). The most important lesson of the above code is how the subscribers are defined: **`C++`** ```cpp C++ // Subscriber ros::Subscriber subObjList = n.subscribe("objects", 1, objectListCallback); ``` A `ros::Subscriber` is a ROS object that listens on the network and waits for its own topic message to be available. When a message is received, it executes the callback assigned to it. We declared a single subscriber to the `objects` topic that calls the `objectListCallback` function when it receives a message of type `zed_interfaces/ObjectsStamped` that matches that topic. ### Conclusion The full source code of this tutorial is available on GitHub in the [zed\_obj\_det\_sub\_tutorial](https://github.com/stereolabs/zed-ros-examples/tree/master/tutorials/zed_obj_det_sub_tutorial) sub-package. Along with the node source code are the `package.xml` and `CMakeLists.txt` files that complete the tutorial package.