Hololight Stream for Unity extension - Meshing
Meshing
AR Foundation provides meshing subsystem to receive and render the real world mesh. This can either be used directly with the subsystem, through AR Foundation’s AR Mesh Manager or MRTK’s Spatial Awareness.
Supported client devices
-
Apple iOS and iPadOS
-
Microsoft HoloLens 2
For more information on meshing in AR Foundation check out Unity’s AR Mesh Manager component page.
Hololight Stream supports Unity’s Meshing Subsystem to retrieve mesh data from the HoloLens 2 client. You can use this mesh data and display it using the MRTK Spatial Awareness System or the AR Foundations Mesh Manager.
Enabling MRTK Spatial Awareness
Setup
-
Open your Unity project.
-
In the scene, select Mixed Reality Toolkit.
-
In the Inspector, go to Spatial Awareness > Enable Spatial Awareness System and check the checkbox.
-
Set the Spatial Awareness System Type to MixedRealitySpatialAwarenessSystem.
-
Set the profile to HololightStreamSpatialAwarenessSystemProfile.
Additional information
For more information regarding the how to edit the observer settings, check out Microsoft’s page, Spatial awareness getting started — MRTK2.
| When using custom profiles for the Spatial Awareness system and the Spatial Awareness observer, it is important to create the correct profile types, as MRTK shows all profiles that use the same base profile type as possible candidates. |
Performance considerations
The HololightStreamSpatialAwarenessSystemProfile by default uses the XR SDK Spatial Observer with its default settings. These settings can be edited in order to specify the functionality of the spatial observer. The following are performance considerations when editing the settings:
-
Update Interval
The update interval will determine how often the observer’s center location is updated, in non-stationary observer mode. When using a lower rate, the meshes will be updated more often and the application will be less performant. -
Observer Extents
A larger bounding volume will display meshes in a larger area. If moving between rooms often, it is beneficial to set a larger volume to avoid constantly adding and removing meshes from the space. If remaining in a single location, smaller volumes are often more performant. -
Level of Detail
This will set the amount of triangles/cubic meter for the mesh generation. A finer mesh will reduce the performance of the application. Setting the value to custom will use the value set in Triangles/Cubic Meter. This is a value between 0.0-1.0, with 1.0 being the highest triangles/cubic meter. -
Display Option
The display option determines how meshes should be displayed. The visible setting will generate display of meshes in the space, however it will be the least performant of the settings. When possible, disable this option and use the meshes for occlusion only.
Limitations
-
Unity currently only supports the Axis Aligned Cube bounding volume. Any other observer shape’s will not be allowed.
-
Recalculate Normals must be enabled to allow Unity to calculate the mesh normals. Without this, the generation will have unexpected behavior.
-
If using a custom Level of Detail, the Unity will only accept a value between 0.0 - 1.0 for Triangles/Cubic Meter.
-
Level of Detail setting is not applicable to iOS Client.
Enabling MRTK Scene Understanding
Setup
-
Open your Unity project.
-
In the scene, select Mixed Reality Toolkit.
-
In the Inspector, go to Spatial Awareness > Enable Spatial Awareness System and check the checkbox.
-
Set the Spatial Awareness System Type to MixedRealitySpatialAwarenessSystem
-
Set the profile to HololightStreamSceneUnderstandingSystemProfile Additional information
For more information regarding the how to edit the scene understanding observer settings, see MRTK Scene Understanding Observer.
Hololight Stream uses its own Scene Understanding Observer instead of Windows Mixed Reality Scene Understanding Observer, so observer needs to be adjusted in the MRTK examples.
| When using custom profiles for the Spatial Awareness system and the Scene Understanding observer, it is important to create the correct profile types. This is because MRTK shows all profiles that use the same base profile type as possible candidates. "HololightStreamSceneUnderstandingObserver" uses its own custom profile type, so it needs to be checked carefully. |
Performance considerations
The HololightStreamSceneUnderstandingSystemProfile by default uses the Hololight Stream Scene Understanding Observer with it’s default settings. These settings can be edited in order to specify the functionality of the spatial observer. The following are performance considerations when editing the settings:
-
Update Interval
The update interval will determine how often the observer’s center location is updated, in non-stationary observer mode. When using a lower rate, the meshes will be updated more often and the application will be less performant. -
Query Radius
A larger radius will display meshes in a larger area. If moving between rooms often, it is beneficial to set a larger radius to avoid constantly adding and removing meshes from the space. If remaining in a single location, a smaller radius is often more performant. -
World Mesh Level of Detail
This will set the amount of triangles/cubic meter for the mesh generation. A finer mesh will reduce the performance of the application.