Face Tracking Native – Kinect 4 Windows v2

Well, with the managed version working in c# (see Face Tracking Kinect 4 Windows V2 - Managed) it should be a breeze to convert the code to a native c++ windows store xaml app :).

Converting an event-based c#/xaml app seemed like a gentle introduction to a more native environment which I hope to get into later. Also, I was asked recently whether face tracking worked in a native app.

There are a few things I’ve become accustomed to in the managed development environment; nuget for one thing and also I’ve been using Reactive extensions in my Kinect projects. A quick Bing-search later and I found this:

So, typing Install-Package rxcpp into the Package Manager Console window caused a short flurry of output which I assume meant the packages were downloaded and referenced within my project. After some short experimentation I found that v2.0.0 doesn’t yet have bindings for WinRT so I rolled back to v1.0.2 with a ‘note-to-self’ to explore this more fully later. Including the headers rx.hpp and rx-winrt.hpp in the projects precompiled header and adding a using namespace rxcpp; enabled me to get started writing the code.

It took me a while to work out the correct syntax for configuring the Kinect event handlers using Rx. I ended up with this:

1. _multiFrameReader = _kinect->OpenMultiSourceFrameReader(FrameSourceTypes::Body | FrameSourceTypes::Color);
2. auto multiFrameObservable = from(rxrt::FromEventPattern<MultiFrameArrivedEventHandler>(
3.     [this](MultiFrameArrivedEventHandler^ h)
4.     {
5.         return this->_multiFrameReader->MultiSourceFrameArrived += h;
6.     },
7.     [this](Windows::Foundation::EventRegistrationToken t)
8.     {
9.         this->_multiFrameReader->MultiSourceFrameArrived -= t;
10.     }))
11.     .observe_on(std::make_shared<rxcpp::NewThreadScheduler>());
12.  
13. _disposables.Add(from(multiFrameObservable)
14. .subscribe([this](const rxrt::EventPattern<MultiSourceFrameReader^, MultiSourceFrameArrivedEventArgs^>& p){
15.     this->OnMultiFrameArrived(p);
16. }));

In the Kinect event handlers when I tried to call Close() on the Frames I got a compile error error C2039: 'Close' : is not a member of 'WindowsPreview::Kinect::MultiSourceFrame' so I used scoping to release the Frames instead.

1. {
2.     auto multiFrame = p.EventArgs()->FrameReference->AcquireFrame();
3.     if (multiFrame != nullptr)
4.     {
5.     }
6. }

Another difference I introduced to the native version was that I couldn’t find a way to schedule rx to deliver the Kinect events via the Thread Pool so I subscribed to the multi frame event from a Thread Pool thread and used the CurrentThreadScheduler. Aside from these minor differences the rest of the code looked pretty much the same. You can find the sample project here http://1drv.ms/1th57Lb.