Magic Leap Motion Tracking

What is Magic Leap?

You may have heard of Magic Leap before (not to be confused with the Leap Motion), but what exactly is it all about? Magic Leap is a company that was launched in 2010. Quietly, without anyone taking much notice, they raised over $540 million from high-profile investors such as Google and Qualcomm. What business is this you might ask?

The augmented reality business.

That in itself might not seem very exciting, after all we’ve seen plenty of augmented reality on the devices we already own, but Magic Leap is promising something more. They are working to perfect photorealistic augmented reality that fully composites with the real world.

In practical terms this means that the company is working on a head-mounted display. The idea is that you’ll done this HMD and see impossibly realistic visions interacting with and fitting into your real environment.

You may be thinking “This sounds a lot like the Hololens.” and you’d be right. The Hololens also uses some very advanced technology to create the illusion that there really is an object on your table or hanging on the wall, when it’s only a digital image. The Hololens has the advantage of being a real product that we’ve seen. It has a release window, it has a price and it’s known to work.

Which probably means that Magic Leap should be worried that Microsoft has stolen its thunder, but before we jump to that conclusion have a look at a demo of the technology released by the company mid-October 2015.

https://www.youtube.com/watch?v=kw0-JRa9n94

According to the description of the video this footage was taken through a Magic Leap device and represent what users can expect to see. If you’ve seen equivalent demo footage from the Hololens it should be clear that this is on a different technical level in terms of imagery.

Object can be occluded by real world object, they reflect ambient light conditions correctly, they present optically (e.g. in focus, blurred) the same as real objects in the scene and when the viewer leans in more detail is seamlessly there. According to the available information the core technique at work here is the production of digital light fields. A light field is a representation of how much light is passing through every point in every direction within a space. Magic Leap seems to be using projected digital light fields to create the stunning visuals that we see in their demos.

If this is a true representation of what Magic Leap can do it may really be revolutionary, but even if we assume that the technology we see works as advertised, how will Magic Leap integrate it into an HMD?

How It Works (As far as we know)

Unlike the Hololens and like the Avegant Glyph, it seems the Magic Leap HMD will be using retinal projection. The digital light field will be projected directly into the retina and calculated in such a way that it composites with the natural light entering your eye from the environment so it all looks equally real to your brain. At least in terms of light physics.

The main point of skepticism about the technology, which almost no one has seen, is that it relies on something called silicon photonics. As you might guess this involves using silicon as an optical medium. Because of its special properties tiny microphotonic silicon components can do all sorts of amazing things with light as it passes through. Modifying the interaction of photons for various purposes. Silicon photonics is already used in fiber optics, but Magic Leap is the first company to attempt creating a light field display using silicon photonics.

This is easier said than done, as the nearly $600 million in funding may not be enough to scale up the technology needed to make everything work. Even intel has run into silicon photonic development issues and they certainly are not short on cash or engineering talent.

If Magic Leap succeeds in building the technology it needs then we’ll see an HMD unlike anything else on the market.

Rather than showing an image to each eye (for stereoscopy) from a tiny display projected onto a medium such as a prism, the Magic Leap HMD will present each eye with a multitude of images, leaving your  eyes to freely focus on different details and angles. In other words it will appear natural and put no strain on either your eyes or sense of comfort. The hope is of course that Magic Leap will get around issues of sickness or discomfort brought on by disjointed imagery.

Whether Magic Leap gets its product to market before it runs out of funding remains to be seen, but the promise and potential of the technology can’t be understated. A working Magic Leap would represent the coming of age for augmented reality. It will go from an interesting toy to a technology with incredible social impact. However until we know more, this remains speculative. Magic Leap needs to produce not only working prototypes (which they have), but also the three Ps: Price, Practicality and Product.