This Robot has Taken its First Baby Step with Zero Computer Simulation

AI technique, reinforcement learning to train the robot dog to walk from scratch in the real world

The University of California, Berkeley researchers have built a unique robot: one that taught itself how to walk. The robot dog is waving its legs in the air like an exasperated beetle. After 10 minutes of struggling, it manages to roll over to its front. The research is remarkable as this robot, a four-legged device reminiscent of a mechanical puppy, learned to walk by itself, without being shown any simulations to instruct it beforehand. Furthermore, inaccuracies in the world models these robots use are very damaging to their performance, and constructing reliable world models takes a lot of time and data.

Reinforcement learning to train the robot dog:

Researchers used an AI technique called reinforcement learning, which trains algorithms by rewarding them for desired actions, to train the robot dog to walk from scratch in the real world. The robot dog is taking its first clumsy steps, like a newborn calf. But after one hour, the robot is strutting around the lab with confidence. 

The common approach in training robots is to use computer simulations to let them grasp the basics of whatever they are doing before making them attempt the same tasks in the real world. Traditionally, robots are trained in a computer simulator before they attempt to do anything in the real world. Teaching robots through trial and error is a difficult problem, made even harder by the long training times such teaching requires.

With reinforcement learning, engineers need to specify in their code which behaviors are good and are thus rewarded, and which behaviors are undesirable. Using this approach, the team successfully trained three other robots to perform different tasks, such as picking up balls and moving them between trays. A new generation of reinforcement-learning algorithms could pick up on the real-working workings, super quickly.