In Pursuit of Making AI Agents with the Right Dose of Curiosity

Researchers are trying hard to ensure that AI agents work with the right dose of curiosity

If you want to pick a top restaurant for dinner. Should you visit your most beloved watering hole or try a new establishment, in the hopes of discovering something superior? But that exploration has a risk. But now thankfully researchers have overcome the problem of AI agents being too "curious" and getting distracted by a given task.   

Really curiosity drives artificial intelligence to explore the world in numerous use cases — autonomous navigation, robotic decision-making, optimizing health outcomes, and more. Machines, in some particular cases, use "reinforcement learning" to accomplish a goal, where an AI agent learns from being rewarded for good behavior and punished for bad. Just like the dilemma faced by humans in selecting a restaurant, these AI models also face difficulties with balancing the time spent discovering better actions (exploration) and the time spent taking actions that led to high rewards in the past (exploitation). Too much curiosity can distract the AI agent from making good decisions, while too little means the AI models will not be able to come up with a good decision.

In the pursuit of making AI agents with right dose of curiosity, researchers from MIT's Improbable AI Laboratory and Computer Science and Artificial Intelligence Laboratory (CSAIL) developed an algorithm that overcomes the problem of AI being too "curious" and getting distracted from the assigned task. Their AI algorithm automatically upscales curiosity when it is required and suppresses it if the AI models get enough supervision from the environment to know what to do.

The AI algorithm was tested on over 60 video games and it was capable to succeed at both hard and easy exploration tasks, whereas previous algorithms have only been able to tackle only a hard or easy domain alone. With this method, AI agents use less data for grasping decision-making rules that maximize incentives.