ChatGPT offers many possibilities, but this AI belongs to OpenAI and cannot be used freely. If you want to create your own applications based on an AI chatbot, discover the best open-source alternatives like OPT, Google PaLM or BLOOM.
Looking back, 2022 was a major year for AI and Machine Learning. In addition to numerous tools for developers and multiple studies published by researchers, we witnessed a boom in Large Language Models.
ChatGPT in particular has created a veritable storm on the Internet. In just a few days, this tool, launched by OpenAI at the end of the year, attracted tens of millions of users imagining all kinds of use cases and applications.
However, ChatGPT also has limitations. Access to this AI is restricted, and users must respect the conditions of use. However, there are open source alternatives for people looking to create their own applications using the language models…
OPT: the open-source GPT from Meta
Developed by Meta, OPT is the main rival of OpenAI’s GPT model. Its name stands for “Open Pretrained Transformer”, and openness is one of its key features.
However, this model has several advantages that enable it to replace GPT. Its performance is similar to that of GPT for Zero-Shot NLP (natural language processing) evaluation, and it even outperforms DaVinci and GPT-3 for hate speech detection.
This comes as no surprise, since one of Meta’s main ambitions is to succeed in censoring hate speech. on its social networks and in the future metaverse. If this functionality is a priority for the applications you wish to develop, OPT may therefore be an excellent selection.
On the other hand, OPT is more environmentally friendly than GPT. The carbon footprint of its drive is seven times smaller than that of GPT-3. Here again, energy efficiency was a priority for Meta, which used its Fully Sharded Data Parallel (FSDP) open-source API and NVIDIA’s tensor parallel abstraction within Megatron-LM. The training consumed around 147 TFLOP per second per GPU on 80 gigabit NVIDIA A100 boards.
L’open source approach to Meta s open-source approach to artificial intelligence. Mark Zuckerberg’s company shares its models, training data, logs and much more. No other tech giant contributes so much to the development of the machine learning sector.
PaLM: the language model of the Google Pathways family
The model PaLM is part of the Pathways ecosystem the architecture used by Google for all its large language models (LLMs). In 2022, several models joined this family, including Flamingo, Gato and PaLM.
These various models have made significant contributions to the field of machine learning, and contributed to the rise of Transformers. With Pathways, Google has demonstrated that LLMs could pave the way for general artificial intelligence…
The performance of these models is astounding, outperforming human beings on certain tasks. However, beyond the models themselves.., the real innovation is the Pathways architecture architecture itself.
First of all, Pathways models are trained on multiple types of data such as text, images and video. This multi-modal training is a major difference from GPT, which is mainly text-based.
Moreover, rather than using the complete architecture for each inference, Pathways uses only a subset of neurons. The models therefore take advantage of the benefits of many neurons for increased performance and a larger number of tasks, while keeping costs to a minimum. This is known as “sparse activation”.
Finally, Pathways models can be multiple types of input for the same task. This makes them far more flexible than other models, which are only able to receive different types of input for different tasks.
The PaLM model has recently been enhanced by reinforcement learning, in the same way as ChatGPT was trained from GPT-3. As a result, this model could outperform ChatGPT thanks to its multi-modal capabilities.
Sphere: Meta’s LLM ready to replace Google
Since the launch of ChatGPT, many experts have suggested that this AI could replace web search engines. In reality, another language model is much better positioned to dethrone Google LLM Sphere, developed by machine learning researchers at Meta.
Its performance is impressive for search-related tasksThis model is capable of browsing billions of documents. Add to this Meta’s other work in the field of natural language processing, and Mark Zuckerberg’s company could well become Google’s great rival.
The Sphere model is capable of traversing a large corpus of information to answer questions, check citations and even suggest alternative quotations that better match the content.
While these capabilities are not enough to replace Google as a general-purpose search engine, they seem ideal for researchers. What’s more, Sphere’s open-source nature allows users to modify the text corpus on which the model is based. This gives it great flexibility. Of all the LLM models, Sphere has the greatest commercial viability…
BLOOM: an LLM to thwart the GAFAM monopoly
BLOOM is an auto-regressive large language model (LLM), trained to continue the text of a prompt entered by the user on large volumes of textual data using industrial computing resources.
Thanks to this training, the model is able to produce coherent text in 46 languages and 13 programming languages. The text produced is so convincing that it is difficult to distinguish it from human-written content.
In addition, BLOOM can perform textual tasks for which it has not even been trained explicitly. All we have to do is present it with text generation tasks.
Thus, BLOOM is very similar to GPT. This is no coincidence, since this model has been created for fight against monopoly in the field of large models. Indeed, over the past few years, the tech giants have conducted a great deal of research using gargantuan computing power that is impossible for other groups of researchers to replicate.
As a result, independent researchers are unable to verify or criticize the studies carried out by these companies. Moreover, Data Scientists tend to take the results of these studies out of context, creating inefficient and expensive pipelines.
The aim of BLOOM is to put an end to this phenomenon. This model is not controlled by GAFAM, and aims to promote free search. If you’re looking for an open-source alternative to ChatGPT, this is an ideal choice, even if its performance is obviously inferior.
Galactica: Meta’s controversial ChatGPT for researchers
Galactica is another LLM developed by Meta, similar to ChatGPT and aimed at researchers. This model has been trained on numerous research projects, and can therefore answer many scientific questions.
In particular, it can help researchers write their studiesto explain a mathematical formula, or to detail the creation of a material. Unfortunately, when the LLM was launched to the public, it caused quite a stir, and Meta had no choice but to remove it from the web.
Internet users had fun hijacking the site, imparting false information and racist and sexist prejudices. In any case, this powerful model could prove very useful for a wide variety of users, and we can expect a new version to be launched in the near future…
You are now familiar with the main open-source alternatives to ChatGPT. Some of these models are more powerful than OpenAI’s, while others take a different approach. In all cases, their openness offers more flexibility and allows for emancipation from the tech giants…