Galactica is the first step toward an ideal scientific neural network assistant

In the past few years, we have observed significant advancement in Artificial Intelligence and large language models (LLMs). From OpenAI's GPT-3, which generates almost accurate texts in a few seconds, to its open-source counterpart BLOOM, magnificent LLMs can be seen one after the other. Now the new development in artificial intelligence is Galactica. Galactica is the first step toward an ideal scientific neural network assistant.

All this impressive advancement has become possible because of the massive amount of data available on the internet and the availability of powerful GPUs. As good as they sound, training an LLM is an extremely costly process, both in terms of data and hardware requirements. We are talking about AI models with trillions of parameters, so it is not an easy task to feed these AI models with sufficient data. However, once you get it successfully, you get a mesmerizing performance out of them. Galactica is the first step toward an ideal scientific neural network assistant. This ultimate scientific assistance will be the interface of how we access knowledge. Galactica will handle the cumbersome information overload process while you focus on making decisions based on the given information.

So, how does Galactica work? Basically, it is a LARGE language model per se, so it contains billions of parameters trained on billions of data points. AS Galactica is developed to be a scientific assistant, the obvious source of training data is research papers. In that regard, over 48 million research papers, 2 million code samples, 8 million lecture notes, and textbooks have been used to construct the training data of Galactica. In the end, a dataset with 106 billion tokens is used. Galactica was used in writing its own paper, so this makes Galactica one of the first AI models that introduced itself. And it can be used to write many more papers in the near future. This was a brief idea about Galactica, the new AI model from Meta designed to help with scientific knowledge retrieval.