Set up Microsoft Mixed Reality Toolkit 3 for Hololight Stream for Unity

Set up Microsoft Mixed Reality Toolkit 3

Hololight Stream works together with Microsoft Mixed Reality Toolkit 3 (MRTK3) to create more immersive mixed-reality experiences. Follow the instructions below to add MRTK3 to your Unity project.

If a project already has Mixed Reality Toolkit 2 (MRTK2) installed, you need to remove it completely before installing MRTK3. This includes Hololight Stream Extension for MRTK2. After completely removing these, you can install MRTK3 and the Hololight Stream MRTK3 extension. For more details about upgrading an MRTK2 project to MRTK3, check out Microsoft’s page, Migration guide from MRTK2 to MRTK3.

  1. Go to Unity’s page for starting MRTK from a new project.

  2. Follow Microsoft’s instructions to download and install MRTK3.

    Newer versions of MRTK3 may work but are not officially supported. Hololight Stream officially supports MRTK3.xx

Install Hololight Stream Extension for MRTK3

Hololight Stream Extension for MRTK3 provides XR functionality for inputs and other features that makes using MRTK3 with Hololight Stream possible.

To install Hololight Stream Extension for MRTK3 import "com.hololight.stream.mrtk3-<version>.tgz" where <version> is the latest version of Hololight Stream using Package Manager. This package is in the "Packages" folder of the Hololight Stream SDK.

MRTK3 will not work with Hololight Stream without the Hololight Stream Extension for MRTK3.

Configure the project to use Mixed Reality Toolkit 3

If you have already configured your project to use Hololight Stream, you can then do the additional project configuration needed to support MRTK3.

Go to Edit > Project Settings > MRTK3 > Profile.

Configure with the added Stream Subsystems. A sample profile can be found in the package. "Packages/com.hololight.stream.mrtk3/Runtime/Profiles/StreamMRTKProfile.asset".

Configure the scene

To configure the scene for Hololight Stream. This will add necessary components for Stream to an existing MRTK XR Rig or a Main Camera and do the necessary modifications, or instantiate the Stream MRTK XR Rig for the scene to use. You need the Hololight Stream Extension for MRTK3.

Select Hololight > Stream > Configure MRTK.

Optional features

Touchscreen input

Add the Touch Controller Prefab as a child to the Stream MRTK XR Rig > Camera Offset GameObject to enable touchscreen input.

Mouse input for Hololight Stream Client for Windows

  1. Add the MRTK Spatial Mouse Controller Prefab as a child to the Stream MRTK XR Rig > Camera Offset GameObject to enable input on Hololight Stream Client for Windows.

  2. Select Stream MRTK XR Rig > Camera Offset > MRTK Spatial Mouse Controller. In the Inspector go to XR Controller (Action-based) > Select Action and select the Stream Mouse/Select Asset Inputs Action found in the package.

  3. Select Stream MRTK XR Rig > MRTK Spatial Mouse Controller > MRTK Spatial Mouse Interactor. In the Inspector under MRTK Spatial Mouse Interactor, change the following propeties to Asset Inputs Actions found in the package:

    Property

    Value

    Mouse Move Action

    Stream Mouse/MouseMove

    Mouse Scroll Action

    Stream Mouse/MouseScroll

  4. Select Stream MRTK XR Rig > MRTK Spatial Mouse Controller > SpatialMouseInteractor in the Hierarchy. In the Inspector set MRTK Spatial Mouse Interactor > Mouse Sensitivity to "0.5".

  5. Select Stream MRTK XR Rig > MRTK Spatial Mouse Controller > SpatialMouseInteractor > MouseCursor > CursorVisual in the Hierarchy. In the Inspector under Sprite Renderer > Flip, make sure there is a checkmark next to X.

MRTK Spatial Mouse Controller has a different behavior than MRTK 2’s spatial mouse. Stream passes mouse movements directly to the MRTK 3 through Input Actions. It is also possible to use these Input Actions for different spatial mouse controller implementations.

Speech input

Select Stream MRTK XR Rig > MRTK Speech in the hierarchy. Make sure the GameObject is enabled.

Gaze

To enable gaze: . Select Stream MRTK XR Rig > Camera Offset > MRTK Gaze Controler in the Hierarchy. Under XR Controller (Action-based with Fallbacks) change the following properties to Asset Inputs Actions found in the package:

Property

Value

Position Action

Stream Gaze/Rotation

Tracking State Action

Stream Gaze/TrackingState
Optionally, to customize gaze the reticle, add the MRTK Ray Reticle Visual component to Stream MRTK XR Rig > Camera Offset > MRTK Gaze Controler > GazeInteractor. Then add the Reticle prefab (or any other custom reticle Prefab) as a child to GazeInteractor. Then in MRTK Ray Reticle Visual reference the reticle Prefab.

Test Mixed Reality Toolkit 3

Enter Play mode to test if you successfully configured MRTK3.

Continue to Set up AR Foundation for Hololight Stream for Unity.