The Leap Motion: Putting you into the virtual world
Head mounted display technology has become pretty adept at immersing us as observers in virtual worlds. We really feel as if we are standing in that imaginary space, looking at events and objects that seem so real that you can reach out and touch them. If you did try to touch them however, you would realize that in this virtual world you are nothing more than a ghost. You can see and hear everything, but you can’t interact with anything. You feel presence, but you have none.
The most common way to interact with virtual reality for consumers today is with the common gamepad or a keyboard and mouse combination. For vehicle or aircraft simulations you might use a joystick or steering wheel system for feeling of authenticity, but none of these input methods feel as natural as simply reaching out with your hand and manipulating an object.
This is where a device such as the Leap Motion comes in. In a nutshell, the Leap Motion is a sensor bar that can scan objects within a particular volume in real time to an incredibly high degree of accuracy. The device is not explicitly meant for embodied virtual reality applications, but virtual reality developers quickly saw the potential of the technology.
The Leap Motion was intended to be an interface between the physical world and the digital one. By placing the Leap Motion in front of your computer monitor you can reach “into” the monitor and interact with something. Perhaps a 3D CAD drawing or some virtual clay meant for sculpting into a 3D model. The device can do this by creating an almost 1-to-1 map of all physical objects (such as your hands) within the volume of space the sensors can see. An amazing feat for an $80 USB device.
How Does It Work?
The Leap Motion is a pretty unassuming little box that wouldn’t look out of place among external hard drives and usb mobile data modems. In terms of hardware the Leap Motion isn’t all that remarkable. Basically it consists of two IR (infrared) cameras and three IR emitters. The emitters bathe objects in infrared light which then reflects back into the two cameras at 300 frames per second. The two cameras then compare these two streams of 2D images and calculate their 3D positions to within just a few millimetres.
That’s about all we know when it comes down to under-the-hood facts. Most of what makes the Leap Motion work is in the software as a product of sophisticated mathematics. The details of which are closely guarded by the company.
How Well Does It Work?
For its price and simplicity the Leap Motion works outstandingly well. There have however been a dearth of applications for the device and certainly no killer application yet. Most reviewers seem to think of the Leap Motion as a way to clumsily turn a computer monitor into a touchscreen. It doesn’t help that a controller that represents a new paradigm for computer interfaces will likely have little or no software support, making it hard to get users onboard with the system.
Professional evaluation of the system has however been positive. One research team compared the Leap Motion to professional tracking systems and, although it cannot yet replace them, indicated that the device might well represent a revolution in gesture-based computer control.
Leap Motion and Virtual Reality
Virtual Reality might well be the killer application that the Leap Motion has been waiting for. It provides a cheap way to add motion tracking of hands directly into the virtual world in real time. The Leap Motion was designed to be placed on a desk, but it was a relatively simple matter to mount it onto the front of an HMD such as the Oculus Rift and modify the software to account for the motions of the user’s head.The Leap Motion’s sensors can map space out to about a meter, which comfortably covers the reach of most users who are not orangutans.
In this demo video you can clearly see the Leap Motion in action, mounted on an Oculus Rift DK2, tracking the user’s hands with very little latency.
Although this sort of virtual reality application was not in the initial marketing push for the Leap Motion, the company wasted no time in embracing the possibilities for virtual reality and its product. Releasing a specialised VR Developer’s Kit and a custom HMD mounting plate.
The OSVR project already includes support for the Leap Motion in the form of software support and hardware mounting in its specifications. The Leap Motion can now also be embedded into other devices, which means that future HMDs may simple have the sensor integrated into its front.
If virtual reality becomes the mainstream phenomenon we believe it will, the technology in the Leap Motion may become very important indeed. It really seems as if this will be the the device’s killer application.
Limitations
As you might suspect there are a number of limitations to the technology. For one thing, with just one sensor only one side of your hands can be seen. Of course, given that hand movement can be calculated from both sides this isn’t a major problem. Secondly, we’ve seen an issue in some demo videos where the Leap Motion becomes confused when the two hands are too close together. According to the research article cited above there is also a loss in accuracy the further you get to the end of the sensor’s range.
The main limitation however, is the lack of haptic feedback. Although your virtual hands can reach out and manipulate the world, you have no way to feel what your virtual hands would feel. Apart from detracting from the immersion, this makes us clumsy in the virtual world. There are already a number of research projects that are combining the Leap Motion with different approaches to haptic (or touch) technologies that let us feel virtual objects. One project, for example, uses ultrasonic sound to simulate the “feel” of an object with targeted sound pressure.
Conclusion
As a first generation product the Leap Motion is revolutionary and remarkable. It seems as if virtual reality and this tracking method are a match made in heaven, but only time will tell if either technology will reach mainstream success.
