Friday, May 20, 2016

Is this what VR cameras of the future will look like?

True VR cameras might be coming sooner than you think.


The 360-degree images we have from current 360-degree cameras are not true VR images.  Although headtracking allows us to look around a 360-degree image, a true VR image should let you move up, down, forward, backward, left or right, and your perspective would change accordingly.  Tracking this type of movement (positional tracking) is necessary to truly fool our senses into believing that we have been transported to another location (presence).


DARPA (Defense Advanced Research Projects Agency), the research agency of the U.S. Department of Defense, in cooperation with Nvidia, has developed a way to capture an environment that may be a preview of how VR cameras of the future could work.  DARPA's "Virtual Eye" uses two cameras that each capture not only light but also depth information.  By combining the data from the two cameras, the Virtual Eye can reconstruct a 3D model of the environment.


Here is a demo image:




On the lower right corner, you can see the layout of the room.  The two blue dots represent the positions of the two cameras.​  The moving gray dot represents the user's point of view. 



On the upper left corner, you see the image captured by the left camera.  On the upper right corner is the image from the right camera.  The image in the middle is the synthesized 3D view from the images from the left and right cameras.



As you can see, the perspective adjusts as the user moves the point of view.  This would qualify as a true VR image, albeit with a narrow point field of view.



What is interesting about the Virtual Eye technology is that it does not require exotic cameras.  Indeed, the demo appears to use two Xbox 360 Kinect cameras (which use infrared to sense depth):


DARPA believes they can implement this technology in the field within just two or three years.


Although DARPA doesn't say it, its technology has implications for consumers as well.  Using a similar approach, it may be possible for consumers to use two 360-degree cameras, with depth sensors, to create a 360-degree 3D model, which could be a true VR image.