Tutorial - Using Spatial Mapping

This tutorial shows how to use a ZED stereo camera to capture a live 3D reconstruction of your environment. The code starts spatial mapping for 500 frames, extracts a mesh, filters it and saves it as an OBJ file.

Getting Started

• First, download the latest version of the ZED SDK.
• Download the Spatial Mapping sample code in C++, Python or C#.

Code Overview

Open the camera

Let’s initialize the camera. In this example, we use HD720 mode at 60 fps (which also has a wide FOV) to make sure camera tracking is reliable. We also select a right-handed coordinate system with the Y axis up, since it is the most common system used by 3D viewing software (Meshlab for example).

1
// Create a ZED camera object
2
Camera zed;
3
4
// Set configuration parameters
5
InitParameters init_params;
6
init_params.camera_resolution = RESOLUTION::HD720; // Use HD720 video mode (default fps: 60)
7
init_params.coordinate_system = COORDINATE_SYSTEM::RIGHT_HANDED_Y_UP; // Use a right-handed Y-up coordinate system
8
init_params.coordinate_units = UNIT::METER; // Set units in meters
9
10
// Open the camera
11
ERROR_CODE err = zed.open(init_params);
12
if (err != ERROR_CODE::SUCCESS)
13
    exit(-1);

Enable positional tracking

Spatial mapping needs position tracking to be enabled first with enablePositionalTracking().

1
PositionalTrackingParameters tracking_parameters;
2
err = zed.enablePositionalTracking(tracking_parameters);
3
if (err != ERROR_CODE::SUCCESS)
4
    exit(-1);

Enable spatial mapping

Spatial mapping setup works the same way as positional tracking: we create a SpatialMappingParameters settings and call enableSpatialMapping() with this parameter.

1
SpatialMappingParameters mapping_parameters;
2
err = zed.enableSpatialMapping(mapping_parameters);
3
if (err != ERROR_CODE::SUCCESS)
4
    exit(-1);

It is not the purpose of this tutorial to go into the details of SpatialMappingParameters class. For more information, read the API documentation.

Run live 3D reconstruction

To start mapping an area, there is no need to call any function in the grab() loop. The ZED SDK checks in the background that a new image, depth and position is available and automatically builds a 3D map asynchronously from this data.

In this tutorial, we simply grab 500 frames, check the 3D mapping state and then stop the loop to extract the resulting mesh.

1
// Grab data during 500 frames
2
int i = 0;
3
Mesh mesh; // Create a mesh object
4
while (i < 500) {
5
  if (zed.grab() == ERROR_CODE::SUCCESS) {
6
    // In background, spatial mapping will use new images, depth and pose to create and update the mesh. No specific functions are required here.
7
    SPATIAL_MAPPING_STATE mapping_state = zed.getSpatialMappingState();
8
    // Print spatial mapping state
9
    cout << "\rImages captured: " << i << " / 500  |  Spatial mapping state: " << mapping_state << "                     " << flush;
10
    i++;
11
  }
12
}

Extract mesh

After capturing a 3D map during 500 frames, we can now retrieve the mesh stored in the Mesh object using extractWholeSpatialMap(). This function will be blocking until the full mesh is available.

1
zed.extractWholeSpatialMap(mesh); // Extract the whole mesh

This mesh can be filtered (if needed) to remove duplicate vertices and unneeded faces. Then we save the mesh as an OBJ file for external use.

1
mesh.filter(MeshFilterParameters::MESH_FILTER::LOW); // Filter the mesh (remove unnecessary vertices and faces)
2
mesh.save("mesh.obj"); // Save the mesh in an obj file

Disable modules and exit

Once the mesh is extracted and saved, don’t forget to disable the mapping and tracking modules (in this order) and close the camera before exiting your application.

1
// Disable tracking and mapping and close the camera
2
 zed.disableSpatialMapping();
3
 zed.disablePositionalTracking();
4
 zed.close();

Advanced Example

To learn how to capture a live 3D mesh of the environment and display it as an overlay on the camera image, check the Live 3D Reconstruction sample code.