Using the Spatial Mapping API

Spatial Mapping Configuration

Spatial Mapping configuration is similar to previous modules. Use InitParameters to set the video mode, coordinate system and units.

1 // Set configuration parameters
2 InitParameters init_params;
3 init_params.camera_resolution = RESOLUTION::HD720; // Use HD720 video mode (default fps: 60)
4 init_params.coordinate_system = COORDINATE_SYSTEM::RIGHT_HANDED_Y_UP; // Use a right-handed Y-up coordinate system
5 init_params.coordinate_units = UNIT::METER; // Set units in meters

The coordinate system determines the axes convention of the spatial map, and coordinate units specify the metrics. Since spatial mapping uses positional tracking to create its map, we recommend using the HD720 video mode at 60fps for optimal results.

Enabling Spatial Mapping

After opening the camera, enable positional tracking using enablePositionalTracking() and spatial mapping using enableSpatialMapping() with SpatialMappingParameters default parameters.

SpatialMappingParameters has two main settings to adjust: resolution and range.

Adjusting Resolution

You can specify mapping resolution manually (in meters) or through the following presets:

MAPPING_RESOLUTION::HIGH: Set resolution to 2cm. Use this setting to map small areas.
MAPPING_RESOLUTION::MEDIUM: Set resolution to 5cm. Good balance between performance and level of detail.
MAPPING_RESOLUTION::LOW: Set resolution of 8cm. Use this setting to map large areas or create a collision mesh.

Mapping on the HIGH resolution setting is resource-intensive, and slows down spatial map updates. Use lower resolutions to get faster updates.

1 SpatialMappingParameters mapping_parameters;
2 mapping_parameters.resolution_meter = 0.03;  // Set resolution to 3cm
3 mapping_parameters.resolution_meter = SpatialMappingParameters::get(MAPPING_RESOLUTION::LOW); // Or use preset

Adjusting Range

You can manually specify the range of depth data to be integrated into the mapping process, or set it through one of the following presets:

MAPPING_RANGE::NEAR: integrates depth up to 3.5 meters.
MAPPING_RANGE::MEDIUM: integrates depth up to 5 meters.
MAPPING_RANGE::FAR: integrates depth up to 10 meters.

Since depth accuracy decreases with distance, mapping range has been limited to 10 meters. Use a careful balance of resolution and range when mapping to maintain a good quality/performance ratio.

1 SpatialMappingParameters mapping_parameters;
2 mapping_parameters.range_meter = 5 ;  // Set maximum depth mapping range to 5m
3 mapping_parameters.range_meter = SpatialMappingParameters::get(MAPPING_RANGE::MEDIUM); // Or use preset

Choosing the Map Type

The spatial map can be of the following types:

SPATIAL_MAP_TYPE::MESH: Generates a mesh with points, faces and edges.
SPATIAL_MAP_TYPE::FUSED_POINT_CLOUD: Generates a colored point cloud. This format takes more memory due to color information.

1 SpatialMappingParameters mapping_parameters;
2 // Set mapping with mesh output
3 spatial_mapping_parameters.map_type = SpatialMappingParameters::SPATIAL_MAP_TYPE::MESH;
4 // or select point cloud output
5 spatial_mapping_parameters.map_type = SpatialMappingParameters::SPATIAL_MAP_TYPE::FUSED_POINT_CLOUD;

Enabling textures

By allowing spatial mapping to save images of the scene, you can export a textured version of the mesh. This option is available only for Meshes, not for Fused Point Cloud, which already has color information attached to each point.

1 SpatialMappingParameters mapping_parameters;
2 mapping_parameters.save_texture = true;  // Scene texture will be recorded

Getting a 3D Map

To start spatial mapping, you just need to call grab(). The Spatial Mapping will ingest new images, depth information, and tracking poses in the background to create the spatial map.

One-time Mapping

The process to map a whole area and save the result is:

Start spatial mapping. Capture the entire area.
When done, use extractWholeSpatialMap() to retrieve the spatial map. This can take some time depending on the size and resolution of the spatial map.

If you choose to create a Mesh you can:

Use mesh.filter() to refine mesh.
Create the mesh texture using mesh.applyTexture().

With both types you can:

Save the map with map.save("filename.obj").

1 // Configure spatial mapping parameters
2 sl::SpatialMappingParameters mapping_parameters(SpatialMappingParameters::MAPPING_RESOLUTION::LOW,
3                                                 SpatialMappingParameters::MAPPING_RANGE::FAR);
4 // In this cas we want to create a Mesh
5 mapping_parameters.map_type = SpatialMappingParameters::SPATIAL_MAP_TYPE::MESH;
6 mapping_parameters.save_texture = true;
7 filter_params.set(MeshFilterParameters::MESH_FILTER::LOW); // not available for fused point cloud
8 
9 // Enable tracking and mapping
10 zed.enableTracking();
11 zed.enableSpatialMapping(mapping_parameters);
12 
13 sl::Mesh mesh; // Create a mesh object
14 int timer = 0;
15 
16 // Grab 500 frames and stop
17 while (timer < 500) {
18   if (zed.grab() == ERROR_CODE::SUCCESS) {
19     // When grab() = SUCCESS, a new image, depth and pose is available.
20     // Spatial mapping automatically ingests the new data to build the mesh.
21     timer++;
22   }
23 }
24 
25 // Retrieve the spatial map
26 zed.extractWholeSpatialMap(mesh);
27 // Filter the mesh
28 mesh.filter(filter_params); // not available for fused point cloud
29 // Apply the texture
30 mesh.applyTexture(); // not available for fused point cloud
31 // Save the mesh in .obj format
32 mesh.save("mesh.obj");

Continuous Mapping

To get a spatial map continuously, a request and update system is available:

Enable spatial mapping.
Send a request to get an updated spatial map using requestSpatialMapAsync(). This will launch spatial map extraction in the background.
Check the request status with getSpatialMapRequestStatusAsync(). When the spatial map is ready, a SUCCESS status is returned.
Retrieve the spatial map with retrieveSpatialMapAsync(sl::Mesh).
Use in your application.

Requesting and retrieving a spatial map is resource-intensive. To improve performance, do not request a new spatial map too frequently.

1 // Request an updated spatial map every 0.5s
2 sl::Mesh mesh; // Create a Mesh object or a FusedPointCloudObject
3 int timer = 0;
4 while (1) {
5   if (zed.grab() == ERROR_CODE::SUCCESS) {
6 
7       // Request an update of the mesh every 30 frames (0.5s in HD720 mode)
8       if (timer%30 == 0)
9          zed.requestSpatialMapAsync();
10 
11       // Retrieve spatial map when ready
12       if (zed.getSpatialMapRequestStatusAsync() == ERROR_CODE::SUCCESS && timer > 0)
13          zed.retrieveSpatialMapAsync(mesh);
14 
15       timer++;
16   }
17 }

Disabling Spatial Mapping

Once the spatial mapping is disabled, you will not be able to retrieve the spatial map anymore. Make sure to extract it before disabling the modules and closing the camera.

1 // Disable spatial mapping, positional tracking and close the camera
2 zed.disableSpatialMapping();
3 zed.disableTracking();
4 zed.close();

Spatial Mapping States

When using spatial mapping, you can check its state by calling getSpatialMappingState().

1 SPATIAL_MAPPING_STATE state = zed.getSpatialMappingState();

When spatial mapping is running correctly, the module will return SPATIAL_MAPPING_STATE::OK. If a system does not allow the ZED SDK to reach the framerate required for a consistent mapping experience, the module will return SPATIAL_MAPPING_STATE::FPS_TOO_LOW. Similarly, if the module reaches the memory limit, it will return SPATIAL_MAPPING_STATE::NOT_ENOUGH_MEMORY. In both cases, the spatial mapping will stop integrating new data into the model, but you can still extract the 3D model.

Code Example

For code examples, check out the Tutorial and Sample on GitHub.