Getting Started The ZED Plugin for Unity allows developers to build mixed reality applications with stereo pass-through or with a third person view. It also provides access to depth sensing, positional tracking and spatial mapping features within Unity. Examples of applications the ZED plugin makes possible are: Building mixed reality (AR/MR) experiences where real and virtual objects are blended together. Displaying AR experiences in Oculus Rift or HTC Vive with stereo video pass-through. Capturing VR applications from a third-person point of view in front of a green screen. ZED Plugin for Unity The ZED Plugin for Unity is available as a ZEDCamera package. It includes assets, scripts, and sample scenes to assist with development. Many of its core features, such as mixed reality, take minutes to implement thanks to its prefabs. The package contains the following subdirectories: Editor: Contains scripts to add functionality to the Unity editor interface. Examples: Contains sample scenes illustrating how to use the plugin. Prefabs: Contains the main prefabs to add ZED camera to a scene: ZED_Rig_Mono and ZED_Rig_Stereo. SDK: Contains files and resources to interface the ZED SDK with Unity. Requirements Unity 5.6.X or higher. Prefabs may not work with earlier versions. DirectX 11, OpenGL Core x86_64 builds only ZED SDK How to Get Started To develop applications in Unity with your ZED, you’ll need the following things: Download and install the ZED SDK Download the ZED Plugin for Unity Import the ZEDCamera.package into Unity Important: After importing the package, you need to set the aspect ratio of your Game Window to 16:9 with a scale of 1x . If the aspect ratio is anything else, the ZED image will appear distorted or blurry in the main window. ZED Rig In the Prefabs folder, you will find a mono and stereo camera rig: ZED_Rig_Mono and ZED_Rig_Stereo. These prefabs are custom MR cameras that replace the regular Unity Camera in a scene. The rig mixes the virtual 3D elements rendered by the Unity camera with the real-world video captured by the ZED. If positional tracking is enabled, the virtual rig automatically moves to match the real-world movement of the device using its own input (no need for an external tracker). This allows you to place virtual objects in the scene and make sure they maintain their real-world position while moving. To add these functionalities to your project, add ZED_Rig_Mono to your Hierarchy view. Use ZED_Rig_Stereo if you need stereo passthrough in a VR headset. Note: When adding ZED Rig to your project, delete the Main Camera from the Hierarchy to avoid interfering with the embedded camera of the prefab. If your use case requires multiple camera outputs in your scene, you may need multiple display windows visible in the editor. ZED Manager The prefab also provides access to ZED Manager, which allows you to configure video, depth and tracking parameters. These parameters are set by default and you usually don’t need to modify them when adding the prefab to your project. After adding a ZED_Rig to the Hierarchy view, click on it to access the parameters of the ZED Manager script in the Inspector window. Camera Resolution: Video mode of the ZED camera. Higher resolutions result in lower fps. See Video Modes. Depth mode: Depth quality mode. PERFORMANCE mode is recommended. See Depth Modes. Motion Tracking Enable Tracking: Enable positional tracking. Disable this option when tracking with an external controller or other tracked object such as a Vive Tracker. See Positional Tracking. Enable Spatial Memory: Enable spatial memory to correct drift during tracking. There can be small pose jumps when a correction is required. See Spatial Memory. Path Spatial Memory: Path to the file that will be loaded and/or saved by the Spatial Memory. Loading an existing file will allow absolute localization in an environment. Leaving blank will not save an area file at the end of a tracking session. Rendering Video Overlay: Add the ZED image in front of the ZED Rig camera (for mixed reality). Camera Brightness: Control image brightness. Use this to darken the real-world video without affecting virtual objects. AR Post-processing: Enable post-processing of the virtual layer to match the real camera characteristics. Rendering Path: Select Forward or Deferred rendering path. Use Forward for better performance in MR. Camera Control: Open a new window to adjust Camera Settings. Using SVO files During application development, it can be easier to work with a recorded video instead of a live video feed. In Unity, you can record and read SVO files by selecting ZED_Rig_Mono or ZED_Rig_Stereo in the Hierarchy panel, then Add Component > SVO Manager in the Inspector panel.