ChatGPT equivalent is open-source now but appears to be of no use to the developers

It seems like the first open-source ChatGPT equivalent has emerged. It is an application of RLHF (Reinforcement Learning with Human Feedback) built on top of Google's PaLM architecture, which has 540 billion parameters. PaLM + RLHF, ChatGPT Equivalent is open-source now, it is a text-generating model that acts similarly to ChatGPT, was provided by the developer in charge of reverse engineering closed-sourced AI systems like Meta's Make-A-Video. It is characterized as a work in progress. To construct a system that can perform nearly every action that ChatGPT can, including email writing and code suggestion, the system combines PaLM, a huge language model from Google, and a method is known as Reinforcement Learning with Human Feedback, or RLHF, for short.

Why this ChatGPT equivalent is of no use to developers?

PaLM + RLHF, is not pre-trained. In other words, the system hasn't received the essential training using example data from the web for it to truly function. A ChatGPT-like experience won't magically appear after downloading PaLM + RLHF; that would need generating gigabytes of text from which the model can learn and locating hardware capable of handling the training demand. Until a well-funded venture (or person) goes to the trouble of teaching and making it accessible to the general public, PaLM + RLHF won't be able to replace ChatGPT today.

The good news is that several additional projects to copy ChatGPT are developing quickly, including one run by the research team CarperAI. The first ready-to-use ChatGPT-like AI model trained with human feedback will be made available by CarperAI in collaboration with the open AI research group EleutherAI, start-ups Scale AI and Hugging Face, and EleutherAI. The non-profit LAION is leading an effort to reproduce ChatGPT using the most recent machine learning methods. LAION provided the initial dataset required to train Stable Diffusion. What will PaLM apps using RLHF be able to do? The performance across activities keeps improving with the model's rising scale, which opens up new opportunities. PaLM can be scaled up to 540 billion parameters. GPT-3, in contrast, only has approximately 175.

ChatGPT and PaLM + RLHF:

Reinforcement Learning with Human Feedback, a method intended to better align language models with what users want them to achieve, is a secret sauce shared by ChatGPT and PaLM + RLHF. RLHF entails fine-tuning a language model using a dataset that contains prompts (such as "Explain machine learning to a six-year-old") matched with what human volunteers anticipate the model to say (such as "Machine learning is a form of AI…"). PaLM is the language model used in PaLM + RLHF. After feeding the aforementioned prompts into the refined model, which produces several responses, the volunteers rank each response from best to worst. The rankings are then used to train a "reward model," which takes the responses from the initial model and sorts them according to preference while filtering for

the procedure of gathering training data is expensive.

Additionally, training is not cheap. PaLM has 540 billion parameters or the components of the language model that were learned from the training set. According to a 2020 study, it might cost up to $1.6 million to create a text-generating model with only 1.5 billion parameters. And it took 384 Nvidia A100 GPUs, each of which costs thousands of dollars, three months to train the open-source model Bloom, which contains 176 billion parameters.

Running a trained model of the size of PaLM + RLHF is also not simple.

A dedicated PC with roughly eight A100 GPUs is needed for Bloom. The cost of running OpenAI's text-generating GPT-3 on a single Amazon Web Services instance, which contains over 175 billion parameters, is estimated to be about $87,000 per year via back-of-the-envelope arithmetic.

Conclusion:

Unless a well-funded venture (or individual) goes through the trouble of teaching and making it accessible to the public, PaLM + RLHF isn't going to replace ChatGPT today.