Robot Transformer 1 to Help Robots Learn from Other Robots

The Robot Transformer 1 (RT-1) is designed to tokenize robotic input and output actions

We've often relied on technology to supplement — and even superpower — our human capabilities. We developed the printing press to help share information, the abacus (and then the calculator) to help us do math, the airplane to help us get from one point to another. In recent years, and specifically in the field of machine learning, we've developed novel ways to process information to power helpful technologies like Search, Assistant, Maps and more. Researchers at Google's robotics arm, Google Research, and Everyday Robots have come up with a way to help robots learn from each other and absorb huge amounts of data to boost performance: the Robot Transformer 1 (RT-1), which sadly is neither an Autobot nor a Decepticon.

"Earlier this year, we worked with Everyday Robots to demonstrate that integrating a powerful language model such as PaLM into a robot learning model could not only enable people to communicate with a robot — but also improve the robot's overall performance," explains Vincent Vanhoucke, head of robotics at Google Research. "This language model made it possible for helper robots to understand several types of requests — like 'I'm hungry, bring me a snack' or 'help me clean up this spill' — and execute them.

"Now, we're using the same architectural foundation as PaLM's – the Transformer – to help robots learn more generally from what they've already seen. So rather than merely understanding the language underpinning a request like 'I'm hungry, bring me a snack,' it can learn — just like we do — from all of its collective experiences doing things like looking at and fetching snacks."

The Robot Transformer 1 (RT-1) is designed to tokenize robotic input and output actions — things like camera feeds, task instructions, and commands to motors — in order to allow for run-time inference efficient enough for real-time control. Trained on a 130,000-episode dataset of more than 700 tasks, gathered from an Everyday Robots fleet over a 17-month period, RT-1 proved capable of significantly improving generalization across new tasks, objects, and environments, boosting its accuracy by observing other robots in action.