Deep Learning in AI Chips

Deep learning is that form of AI which excels in incorporating the human brain that ultimately aids in better decision-making capabilities. There are numerous applications that rely on deep learning. One such application that garnered attention from everyone across is its incorporation in AI chips.

Jeff Dean, an American computer scientist and also Google's brain director had mentioned how Google would be using artificial intelligence to advance its internal development of custom chips about a year ago. This would ultimately pave the way for accelerating its software. Dean spoke about how machine learning could transform how we look at things and be used for some low-level design decisions, known as "place and route." In this, the chip designers would use software to determine the layout of the circuits that form the base of the chip's operations. This is so much similar to designing the floor plan of a building.

No wonder, there have been instances where AI has proved to work better than humans. So much so that organizations are considering it to be their first preference. The same is the case with deep learning. Jeff Dean emphasized on how deep learning can in some cases make better decisions than humans about how to layout circuitry in a chip.

Recently, Google came up with one of its research projects called Apollo, which represents a fascinating development. This moves past what Jeff Dean had spoken about a year ago. But what could be seen in Apollo is that the program is performing architecture exploration. Now, this is contrary to what Dean had mentioned back then. Apollo doesn't talk about a floor plan, to be specific. Apollo is aimed at running different approaches in a methodical fashion and telling what works best.

This "Architecture exploration" is much higher-level than place-and-route and is also where a higher margin for performance improvement exists. It works on different chips and there's an exclusive team that's involved in developing chips for AI, known as accelerators.

Now the real task is to design AI chips. Since Apollo focuses on neural networks, the architecture would revolve the same. And this is why – it goes linear algebra, simple mathematical units that perform matrix multiplications and summing the results. Well, definitely not an easy task to proceed with. The search is just not limited to a few parameters. Areas pertaining to how many of the math units, called processor elements, would be used, and how much parameter memory and activation memory would be optimal for a given model, to name a few have to be covered.

Simply put, the architecture employed in Apollo is such that it is able to figure out how well different optimization approaches will fare in chip design. Despite the fact that chip design is being affected by the new workloads of AI, the process of designing the chip might have an impact on the neural network.