HoloLens Voice Commands

 

This is the fourth part of a series see /3d/blender/hololens/modelling/2016/11/11/hololens-end-to-end-in-unity.html</a> Steps to create the demo project that I used in my Future Decoded UK 2016 presentation

To get set up with the HoloToolkit in Unity see https://mtaulty.com/2016/11/10/hitchiking-the-holotoolkit-unity-leg-1/

KeywordManager

This is all about the KeywordManager from the HoloToolkit; this enables you to define voice commands and have them dispatched at runtime to an object and function that you specify in the Unity editor. So, the first step is to locate the KeywordManager script in the HoloToolkit (Tip: use the project pane search box) and drag the KeywordManager script onto your Managers game object (see the GAZE post). Then selecting the Managers object in the Unity Editor will show the following in the Inspector panel:

 

First, set the Keywords And Responses Size value to 1

Then add your keyword (I used “Reset Stage” in my demo)

Click the ‘+’ symbol to add a receiver object reference and function to be called automatically when the voice command is detected.

In the input box which has None (Object) you can drag an object from the scene hierarchy panel and it will be set as the object reference. For my demo I dragged the StackedBottles object which is the parent to the bottles after having added a function called ResetBottles to a script StackedBottles.cs which I added to the StackedBottles game object.

Which calls through to the BottleScript.cs script with the following code

So, the position and rotation of the individual bottle is cached up front and the Reset function will restore the initial position and rotation.

Note. The call to GetComponent<Rigidbody>().Sleep(); will prevent the physics engine from running whilst the bottles are being reset enabling us to position them all before they all start to fall over.

Also note that there is a dropdown with the options Off, Runtime Only, Editor & Runtime – and… this works in the Unity editor if you select Editor & Runtime implemented, I guess with older .NET APIs to do the speech recognition.

This shows the result:

All resources used are available on Github https://github.com/peted70/fd-holodemo (see the VOICE branch)