Virtual Reality is Perfect for Training Our Robot Overlords

Have you ever thought how those robots that build our cars or vacuum our floors learn to do their job properly? You may think that someone simply sits down and programs all they need to know into them. Certainly there’s a lot of programming that goes into even a simple robot, but it turns out that there are many ways we need to train robots.

For example, apart from direct programming you directly guide a robot by hand to perform a task. It records what you manipulate it to do and can then just play it play it back. Have a look at this video of someone training a welding robot to do a job. (Video Credit: Mechatronic)

https://www.youtube.com/watch?v=OG5FNRZ2XT0

It may seem tedious (and it is), but once you’re done that training can simply be copied to other robots of the same type who have to do the same job. Of course, these days we want robots to do more than just simple repetitive tasks, so machine learning based on the analysis of data has also enabled amazing things.

Training Room

The problem is that you need a lot of time and space to either directly train a robot or allow it to learn something  for itself. If we look at companies like Boston Dynamics or the man contestants that brave the DARPA robotics challenges, they usually set up test courses and let the machine painstakingly improve itself.

The thing is, to the digital “brain” of a modern robot there’s no difference between “real” and “simulated”. It ends up creating a digital representation of the environment anyway and acts within its own internal simulation based on sensory data from the outside.

So why not just cut out the middleman? Why not let robots learn within fully virtual simulations, with simulated VR robotic bodies?

Open Sesame

That’s exactly the question the folks at Open AI both asked and successfully implemented. A VR robot watches a human do a task within that VR world and can then imitate it after only seeing it once.

In the video we see the robot easily stack blocks after seeing the VR human do it first. This is the way that humans start their path to learning, we’re told. Young children mainly learn how to do things by watching it being done by others. Most of our learning is “vicarious” or observed learning.

How Did they Do It?

It’s all down to those wonderful neural nets. A neural net is a type of artificial learning system that reconfigures itself in response to new information until it does something right. As the name suggests, it’s based on how our own neurons strengthen or weaken their bonds in response to success or failure, but computer neural nets are not directly comparable to human neural networks.

This system from Open AI uses two of these networks. On receives the data from a camera (in VR or the real world) and figures out what’s being demonstrated. That is then fed into an neural network dedicated to figuring out how to mimic what has been seen. It then takes control of the arm and tries to do just that.

Smarter Every Day

This is a great step forward for the process of training our robot friends, who in the coming years will become a common sight in all walks of life. This could also mean that training robots could become faster and cheaper, since you can create virtual learning spaces that are difficult to simulate physically and impractical to access for real. Think about building that haven’t been built yet, the bottom of the ocean or outer space. If you can simulate these environments for demonstration purposes your robot can learn without even building the robot!