Windows Phone Mobile VR (Stereoscopic Rendering + Lens Distortion in Unity 3D)

As mentioned in the previous post Windows Phone Mobile VR (Gyro Head Tracking in Unity3D) I will look at stereoscopic rendering and removing the lens distortion effect so that I can run the demo inside my ViSR headset. See the kickstarter campaign here https://www.kickstarter.com/projects/1122924030/visr-virtual-reality/description . This is a great mobile vr headset – definitely worth backing the kickstarter if you are interested in mobile vr.

From my previous post

“The basic idea behind how this works is that each eye receives a 2d image of a 3d perspective view which are rendered offset from each other. The brain assembles these projections from the retina giving the illusion of depth.”

“The lenses allow a much wider field of vision but they also distort the image. These factors result in the need to render your 3D scene twice, side-by-side each from a slightly different viewpoints and also to render it in such a way as to ‘undo’ the lens distortion and keep any straight lines looking straight.”

The two projections will be created using ‘stereoscopic rendering’ and barrel distortion will be used to correct the lens distortion.

The first thing I did, of course was to look around the internet to see if I could find a mobile VR SDK which already had windows phone support – ideally a Unity extension that I could drop onto my character controller which would make everything just work. My search didn’t turn up with anything directly applicable so instead I looked around to see if I could find anything close that I could repurpose. I found a few projects with Android/iOS support but decided to use ALPS VR which seemed to have everything I needed aside from Windows Phone Gyro support (luckily, I have a solution for that in my previous post :-))

First, I downloaded the ALPS unity package and added it to the scene from https://github.com/peted70/head-tracking-unity (from my previous post). I created a new git branch ‘stereoscopic’ and added the ALPS unity package to the Unity project. At this point the project won’t compile for windows phone so the next step is to fix this up;. mainly this consisted of #ifdefs and stubbing things out. The Unity package contains a prefab called ALPSCamera which contains everything required to generate the scene hierarchy required for the stereoscopic rendering and barrel distortion correction at run time. So I dropped it into a scene with the same terrain that I used before.

The initial scene now looks like this:

ALPSController

The main script ALSPController is attached to the prefab and creates the scene hierarchy shown below:

The viewer’s head is modelled with the Head GameObject to which is attached the HeadTrack script from my previous post so that we can use Windows Phone. Two GameObjects are created as children of the head and these represent the view from each eye (CameraLeft and CameraRight). Each ALPSCamera script creates and draws a mesh which undoes the barrel distortion. The meshes are rendered side-by-side onto the main texture as can be seen below:

  

So, let’s take a look at the scripts attached to the prefab:

The ALPSController script has an editor in which you can choose some presets for the device you are using; there are presets for various mobile VR headsets. There is a list of associated parameters:

IPD – this is a measurement of the distance between the pupils of the viewer

ILD – this is a measurement of the distance between the lenses

Barrel Distortion parameters – this allows input of the parameters which define the barrel distortion and should be set to match the lenses in use

Chromatic correction

(see http://en.wikipedia.org/wiki/Distortion_(optics) for a description of barrel distortion and chromatic aberration).

A shader is used on the textures to correct the chromatic aberration.

ALPSNavigation

This provides translation for the viewpoint and uses a scheme which moves dependent on the angle of the head. I used an on-screen button on the phone in my previous post but obviously this would not be available when using a headset. I’d like to explore using a bluetooth connection to another phone or gamepad to provide the movement.

Character Controller

This is the standard Unity Character Controller and provides the code for actually moving the viewpoint.

 

Here is the result running on a Lumia 930 with the ViSR headset:

 

The sample Unity project is on Github here https://github.com/peted70/head-tracking-unity in the ‘stereoscopic’ branch.