OPENFRAMEWORKS + KINECT 4 WINDOWS V2 (II)

In my previous post I showed how to get an environment set up with Openframeworks and Kinect V2.0 for Windows Store. I’d like to elaborate on that a little and run through a demo app I recently created. Since these posts are based on the MSOpenTech fork of Ofx which works with Windows Store we are restricted to using OpenGL ES as this is what is currently supported by Project Angle. What this means in practice is that if you find an oFx addon that you want to use you need to make sure that it doesn’t call methods unsupported by OpenGL ES. This includes things like glBegin..glEnd immediate mode rendering syntax – this is replaced with vertex buffer rendering. If an addon has used this style then it can’t be used with Windows Store currently. Another option here would be to create an oFx desktop app with Kinect integration – more on that in a subsequent post…

Anyway, whilst leafing through the interesting addons I could use I found Box2D which is a 2d physics engine which I thought might work well in a demo.

In the build of the oFx project generator that I am using (from the universal branch here https://github.com/MSOpenTech/openFrameworks) the addons feature doesn’t seem to work. To work around this I used a different fork of oFx from here https://github.com/liquidzym/openFrameworks/tree/VS2013 which has support for Visual Studio 2013 and I added the addon and then I copied the entries from the Visual Studio project files across into my Windows Store projects.

Using that method I got Box2D into my app. From there I could define a Box2D world with physical properties and boundaries and I could introduce rigid bodies into the world. I chose to populate the world with differently-coloured circles, I set the boundaries to be the edges of the app and I used the Kinect body data to make my hands attraction points for the circles.

Here’s the code to set up the world:

1. _box2dWorld.init();
2. float w = ofGetWidth();
3. float h = ofGetHeight();
4.  
5. _box2dWorld.createBounds(0.0f, 0.0f, w, h);
6. _box2dWorld.setGravity(0, 1.0f);
7. _box2dWorld.createGround(0.0f, h - 300.0f, w, h - 300.0f);

And then for setting up a boundary at each side of the screen:

1. b2BodyDef bd;
2. auto wallleft = _box2dWorld.world->CreateBody(&bd);
3.  
4. b2EdgeShape shape;
5. shape.Set(b2Vec2(0.0f / OFX_BOX2D_SCALE, 0.0f / OFX_BOX2D_SCALE), b2Vec2(10.0f / OFX_BOX2D_SCALE, h / OFX_BOX2D_SCALE));
6. wallleft->CreateFixture(&shape, 0.0f);
7.  
8. b2BodyDef bd2;
9. auto wallRight = _box2dWorld.world->CreateBody(&bd2);
10.  
11. b2EdgeShape shape2;
12. shape2.Set(b2Vec2(w / OFX_BOX2D_SCALE, h / OFX_BOX2D_SCALE), b2Vec2((w - 10.0f) / OFX_BOX2D_SCALE, 0.0f / OFX_BOX2D_SCALE));
13. wallRight->CreateFixture(&shape2, 0.0f);

I made a function to create the circles:

1. void ofApp::createCircle(float x, float y)
2. {
3.     float w = ofRandom(10, 60);
4.     ofColor col(ofRandom(0, 255), ofRandom(0, 255), ofRandom(0, 255), 150);
5.     _circles.push_back(ofPtr<ofxBox2dCircle>(new CustomParticle(col)));
6.     _circles.back().get()->setPhysics(3.0, 0.83, 0.1);
7.     _circles.back().get()->setup(_box2dWorld.getWorld(), x, y, w);
8. }

And set up attraction points at the location of the hands:

1. if (_bodiesUpdated)
2. {
3.     for (auto body : _bodies)
4.     {
5.         if (body->IsTracked)
6.         {
7.             auto hl = body->Joints->Lookup(JointType::HandLeft);
8.             auto csp1 = _coordinateMapper->MapCameraPointToColorSpace(hl.Position);
9.             auto h2 = body->Joints->Lookup(JointType::HandRight);
10.             auto csp2 = _coordinateMapper->MapCameraPointToColorSpace(hl.Position);
11.             for (int i = 0; i<_circles.size(); i++)
12.             {
13.                 _circles[i].get()->addAttractionPoint(ofVec2f(csp1.X, csp1.Y), 3.0f);
14.                 _circles[i].get()->addAttractionPoint(ofVec2f(csp2.X, csp2.Y), 3.0f);
15.             }
16.         }
17.     }
18. }

Note the use of the Kinect SDK’s coordinate mapper to map the hand locations from absolute 3d positions to 2d points in my physics world.

It is mildly amusing to make the circles chase you as you swing your hands around but not difficult to let your imagination run riot thinking about the possibilities here. My demo was put together with a few lines of code – that’s the power of OpenFrameworks and other similar frameworks – they include a host of creative elements without the need to reinvent everything.

The source for this is on Github - https://github.com/peted70/kinect-ofx-box2d-circles