Model Search: A Platform for Finding Optimal ML Models

As known to many, Google has recently released Model Search which is an open-source platform. This caters to developing efficient and best machine learning models automatically. Rather than focusing on a particular domain, Model Search is domain agnostic and flexible beyond imagination. Well, not just that. To our surprise, it is even capable of finding just the right architecture. With this, it will best fit a given dataset and the associated problem. At the same time, it is mastered enough to minimize the time that goes behind in coding, the effort as well as the resources that are put in.

Model Search is built on Tensorflow. It is flexible to the extent that it can run either on a single machine or in a distributed setting. This feature does set it apart from the rest, without any doubt. It is equipped with multiple trainers, a search algorithm, a transfer learning algorithm and a database as well that aims at storing the various evaluated models.

The Architecture of Model Search

Talking about the architecture of Model Search, it is based on four foundational components: They are –

1. Model Trainers:As evident as it can get, these components cater to training and evaluating the various models asynchronously.
2. Search Algorithms: With this component of a search algorithm, it is possible to select the best trained architectures. It doesn't end there. There's also an option for the user to add some "mutations" to it which can be sent to the trainers for further evaluation.
3. Transfer Learning Algorithm:Model Search boasts of using transfer learning techniques such as knowledge distillation. This further brings in an advantage of reusing knowledge across different experiments. Model Search enables this using two ways – knowledge distillation or weight sharing. The former allows improving the accuracy by adding a loss term that matches the predictions of the high-performing models. On the other hand, the latter bootstraps some of the network's parameters from previously trained candidates.
4. Model Database:This is where the results of the experiments can be persisted in such a way that it can be reused on different cycles.

What Makes Model Search So Unique?

• Now, that one aspect which makes people look forward to Model Search is its ability to run training and evaluation experiments for AI models in an adaptive and asynchronous fashion. This ability paves the way for the trainers to share the knowledge that they've gained from their experiments. The working is such that at the beginning of every cycle, the search algorithm closely monitors all the completed trials. After this, what follows is deciding what to try next. This platform makes use of beam search while deciding what is to be tried next. The next step that follows is to invoke mutation and assign the resulting model back to a trainer.
• When the users go for a Model Search run, they are in a position to compare the many models found during the search. Well, there's more to this. The platform also allows the users to create their own search space. With this, it is possible for them to customize the architectural elements in their models.
• The researchers have also claimed that Model Search improves upon production models with minimal iterations.

All in all, The Model Search Code aims to provide the researchers with a flexible, domain-agnostic framework to develop the most efficient and best machine learning model. Also, the framework is so powerful that it can build models with state-of-the-art performance. It also has the capability to deal with well-known problems when provided with a search space composed of standard building blocks.