If Artificial Neural Networks Sleep Better, They Train Better

Artificial Neural Networks need to sleep peacefully to function efficiently as humans do

By the end of the decade, experts believe that humans will achieve a new momentum of progress in technology, and at the heart of this progress will be artificial intelligence. Researchers all over the world are working on accelerating AI research and development from its nascent stages to its most advanced form where it will no more need the help of humans to operate and can, in fact, work synonymously with humans to make our lives easier. AI's industrial growth is speculated to explode and its impact on businesses and society will begin to emerge. Coming to the present condition of the technology, there are still miles to go in terms of research and developments that will enable AI to function independently. Scientists are looking for new ways to help technology and its applications retain more information to make it smarter than us. So, in attempts to do the same, researchers have made a new discovery claiming that artificial neural networks can retain information much better if it sleeps better!

Researchers at the University of California, San Diego School of Medicine claim that AI is just like humans. A human brain is extremely busy when it sleeps, repeating what we have learned during the entire day, and the same can be said for artificial neural networks as well. Maxim Bazhenov, Ph.D. and a professor of medicine and a sleep researcher at the University of California has demonstrated how sleep builds rational memory, the ability to remember arbitrary or indirect associations between objects, people, or events, and protects us against forgetting old memories. In a nutshell, the brain works in mysterious ways, but however it performs, it can also help artificial neural networks in the same way order to help machines retain the same amount of knowledge and make them as productive and efficient as humans.

Catastrophic Forgetting Have Been Making Knowledge Retention Difficult in AI

Basically, artificial neural networks leverage the architecture of the human brain to improve numerous technologies and systems, starting from basic science and medicine to finance and social media. Infact in certain ways, they have achieved superhuman performance through computational speed, however, these ANNs fail in a major aspect. They fail to fail to learn sequentially. It is a phenomenon called 'catastrophic forgetting' where new information overwrites any previous information or training. And if we consider the training aspect, catastrophic forgetting in AI can lead to major setbacks for researchers working on a major mission.

To eradicate this issue, scientists have used spiking neural networks that can artificially mimic natural human neural systems. So, instead of continuously inputting information inside the machine, it is transmitted as discrete events (spikes) at certain points in time. They discovered that these spiking neural networks worked even better if it was paired with occasional line-offs, which resemble closely to a machine sleeping like humans. By deploying these approaches, the team of scientists found out that the networks can learn continuously and can avoid catastrophic forgetting on their own.

Conclusion

AI's evolution is set to embark on an exponential growth journey through advanced computing power, deploying cloud storage and memory systems, and the acceleration of the smart device ecosystem. It is only a matter of time before humans will be able to make the best use of AI. It seems like eradicating the catastrophic forgetting phenomenon among machines is the first to attaining efficiency and growth.