Hololight Stream for Unity extension - Video passthrough

Video passthrough

Hololight Stream supports passthrough XR on the devices listed below.

In passthrough mode, every black object in the scene with an alpha value of 0 will not render. This means that objects or skyboxes with an alpha of 0 will not be displayed and instead replaced with the user’s surroundings.

Refer to Unity’s documentation for setting up different render pipelines.

Additional package requirements

  • None

Supported client devices

  • Apple Vision Pro

  • Apple iOS and iPadOS

  • HTC VIVE devices

  • Lenovo ThinkReality VRX[1]

  • Pico 4 Ultra

  • Meta Quest devices

  • Apple Vision Pro

Alternate render pipelines

For different render pipelines check the Unity guides. Hololight Stream requires the background color to be solid black with an alpha value of 0.

Example

  1. Create a 3D URP Unity project (the rest of this section has been done with the 3D Sample Scene (URP) available in the Unity Hub).

  2. Follow the instructions in [Hololight Stream examples setup].

  3. Open Package Manager. Import the URPAlphaPassthrough sample from the Hololight Stream Examples package.

  4. Go to Assets > Settings > Universal Renderer (Univeral Renderer Data) > Post-Processing > Enabled. Set Data to URPPostProcessDataWithAlpha (Post Process Data).

    Screenshot of Unity Editor Inspector panel while Universal Renderer is selected. The post-processing section is highlighted.

  5. Go to Assets > Settings > UniversalRP-High Quality > Quality. Uncheck the box next to HDR.

    Screenshot of Unity Editor Inspector panel while UniversalRP-High Quality is selected. The HDR checkbox is highlighted.

  6. Right click the Scene section and select XR > Convert Main Camera to XR Rig to use the scene with an XR device.

  7. Go to XRRig > Camera Offset > Main Camera and in the Rendering sectio, set:

Main Camera rendering settings

Property

Value

Post Processing

Box checked

Anti-aliasing

No Anti-aliasing

Culling Mask

Default

+ Screenshot of Unity Editor Inspector panel while Main Camera is selected. The post processing, anti-aliasing, and culling mask options are highlighted.

  1. Go to XR Rig > Camera Offset > Main Camera and, in the Background section, set:

    • Background Type to Solid Color

    • Background to zero in R,G,B and A

  2. Enter Play mode and connect with Hololight Stream Client.

High Definition Render Pipeline

By default, the High Definition Render Pipeline (HDRP) does not render alpha information.

For more information, check out Unity’s documentation, Alpha channel configuration.

To enable alpha rendering, follow Unity’s steps below:

  1. Open the HDRP Asset in the Inspector.

  2. Go to Rendering > Color Buffer Format.

  3. Select R16G16B16A16.

Troubleshooting

If you are having trouble getting passthrough to work, check the following:

  • Check that you enabled the passthrough setting on the client device.

  • Check that both the client and server are using the same color space.

  • Check that the Main Camera’s background is a solid color with alpha value of 0.

Limitations

  • Hololight Stream does not support transparency. Transparent objects will appear opaque. The passthrough image won’t appear through them.

  • To make the background transparent, make it black with an alpha value of 0.

1. Hololight Stream Client for Lenovo VRX and Magic Leap 2 is in maintenance. This means that these versions of Hololight Stream Client is either feature complete or deprecated.