Hololight Stream for Unity extension - Speech recognition

Speech recognition

Speech Recognition with Mixed Reality Toolkit

Speech Recognition allows you to interact with scene objects using a list of defined keywords.

Hololight Stream supports speech recognition for Microsoft HoloLens 2 when using the required Speech and Dictation Providers for MRTK. You can use dictation recognition to input a continuous sentence to an object in the scene, for example an input field.

When using Hololight Stream Extension for MRTK2 or MRTK3, speech and dictation recognition are enabled default.

Additional package requirements

  • Mixed Reality Toolkit 2 and Hololight Stream Extension for MRTK2
    or
    Mixed Reality Toolkit 3 and Hololight Stream Extension for MRTK3

Supported client devices

  • Microsoft HoloLens 2

Configure the scene

To use Speech and Dictation, exchange the Windows Speech Input and Windows Dictation Input in the MRTK providers with Isar Speech Input and Isar Dictation Input respectively.

Example

An example on how to use Speech and Dictation can be found in the MRTK Examples Package. Guides on the respective examples can be followed and then migrated into Hololight Stream by applying the steps above.

After following the MRTK Extension Scene Configuration Step, the MRTK configuration profile will be overwritten, changing the SpeechCommandsProfile. Before running the example, set the Input > Speech configuration to Speech.MixedRealitySpeechCommandsProfile.

Speech Recognition without Mixed Reality Toolkit

Speech recognition features can also be used without MRTK. For this, IsarKeywordRecognizer and IsarDictationRecognizer classes can be used.

Additional package requirements

  • None

Supported client devices

  • Microsoft HoloLens 2

Keyword recognition

"IsarKeywordRecognizer.cs" is responsible for keyword recognition.

Dictation Recognition

"IsarDictationRecognizer.cs" is responsible for dictation recognition.

To use both recognition types at the same time, user should make sure to stop one before starting another. As the keyword recognition is done continuously and the dictation recognition is done with a timeout, manual stopping is only needed for keyword recognition. To check the status of the recognizers, Status property can be used. Default status is the stopped recognition. The Hololight Stream examples package includes an example on how to use this class.