John Carmack Reveals 5K VR Video
Despite smartphone screens reaching ludicrous resolutions and fantastic specifications, there are very few 3D-rendered applications that can take advantage of it. So most 3D games and other real-time 3D applications are produced at a lower resolution and then scaled up to the native resolution of the screen. Even then, the absolute rendering ability of top-class phone hardware still doesn’t come anywhere near what’s possible with even mid-range PC hardware. It’s simply the limits of current technology and physics. Without active cooling, space and power there is only so much you can do with a mobile system on a chip.
Famed programming genius John Carmack is thinking of ways to bring that high-end VR experience to mobile VR solutions. He’s especially motivated since the company he works for as CTO, Oculus, has recently released the Oculus Go. This is a standalone headset that uses mobile electronics to provide VR functionality on the, well, go.
Cheating in a Good Way
Carmack’s solution is to pre-render extremely high-quality versions of the VR experience as video. Obviously, this means that only non-interactive VR experiences are possible. Still, it’s those non-interactive experiences where visual quality matters the most. If you’re shooting down aliens or driving a race car, you have less time to take in the fine details.
What Carmack has proposed is VR video with an effective 5K resolution running at a smooth frame rate. That’s not quite as taxing as rendering it all in real time, but even modern smartphones can choke on video at that resolution. Fear not, since John Carmack revealed his solution to providing extreme, 3D VR video on mobile platforms for significantly less computing power than usual.
Playing it Smart
The goal was to provide 360-degree, stereoscopic video at 60 frames per second with smooth motion. The traditional way to render 360-degree video is to decode the whole thing and let the user look at the bits that interest them. That’s pretty wasteful and the drain on total compute power limits the resolution and framerate. Not to mention 3D video is completely out.
Carmack and his team have developed “view dependent” decoding of the video. In other words, the bits you are looking at get decoded at insane quality levels, with the rest filled with lower-quality placeholder video. When you look at another part of the scene, the decoding smoothly switches that part to high quality. Hopefully without you noticing.
The technology was demonstrated with a pre-rendered version of the award-winning Henry VR short. The net result is a VR experience almost as good as the PC version. The even better news is that you can experience this for yourself on either the Gear VR or Oculus Go. The mobile version of Henry using this technology is available in the Oculus store now.
Be sure to read Carmack’s detailed explanation of how all the elements are stitched together. It’s much more impressive than this simple explanation suggests!
