Conversation AI breakthrough will be a new chapter in transforming robotics

Researchers at the University of Tokyo showed that Alter3 doesn't need to be explicitly programmed for every action; it can do things like take a picture, play an instrument, or act weirdly when talking in everyday language. GPT-4 (which supports ChatGPT) works the same way. When it's needed, it can take a picture with DALL-E 3 or understand what people say in whatever language they're speaking.

These conversational AI breakthroughs narrow the communication gap between conversational engagements and physical robots, which usually rely on hardware-driven programming for fine-grained motion control. The scientists translated high-level directives into Alter3-comprehensible instructions. The robot progresses from basic shimmies to more coordinated movements in the same manner that people naturally pick up new motions. Its users may instruct Alter3 to strike certain positions and help it recognize nuances, such as unique dancing moves.

Commands like "pretend you're a ghost" are translated by GPT-4 into a series of Python instructions that Alter3 carries out.  Alter3's lower body is stationary, so its range of motion is restricted to arm motions. The team believes that its methods might be applied to other robots with few to no modifications. According to the experts, this reportedly improves the ability of humanoid robots to understand conversational commands and respond contextually with realistic body language and facial emotions.

This is not the first attempt to combine LLMs and robots. Nvidia also used GPT-4 to develop an AI training system known as Eureka. Eureka trains robots to complete tasks faster than the conventional method, using an autonomous training environment and self-generating reward algorithms. Eureka is similar to the un-OpenAI system used by a team at the University of Michigan to win the inaugural Alexa SimBot Challenge using a virtual robot called Seagull. Also, Alter3 is not the first robot to use a relatively un-optimized version of OpenAI's models. Robotic AI software developer Levatas embedded ChatGPT in one of Boston Dynamics 'Spot' robot dogs. They connected it to Google's synthetic voice API, allowing the metal canine to respond to spoken, informal questions and commands.