Racist and crazy, Meta’s new AI euthanized after 2 days

Meta’s Galactica intelligence was supposed to produce scientific articles based on millions of studies on any subject. Unfortunately, Internet users succeeded in making it lose its pedals in less than two days. Faced with the absurd and racist writings generated by its AI, the American firm had to remove it from the web immediately…

November 15, 2022, Meta unveiled a demo of Galactica a large language model designed for “ store, combine and reason about scientific knowledge “.

The initial aim was to speed up the writing of scientific literature. However, when testing this AI, malicious users discovered that it could also generate totally absurd texts. After a few days of controversy over this AI, Meta finally decided to withdraw the demo from the web.

Galactica: an AI trained on 48 million scientific articles

Meta’s AI Galactica was designed to write scientific literature. Its authors trained it on ” a large organized corpus of humanity’s scientific knowledge “including 48 million articlestexts and notes from scientific websites and encyclopedias.

In total, this AI was equipped with 120 billion parameters. Originally, Meta’s AI researchers were convinced that this high-quality data would enable equally excellent production.

In particular, the model was to synthesize scientific knowledgelike a dedicated search engine. For example, it could be used to summarize all studies on Covid or quantum computing without having to browse through hundreds of thousands of articles on PubMed or arXiv.

From Tuesday November 15, visitors to to the Galactica could enter “prompts (text descriptions) to generate documents such as wiki articles, reading notes or answers to questions.

At least, that’s what the examples on the website promised. The model was presented as “ a new interface for accessing and manipulating what we know about the universe “. Unfortunately, things didn’t go as planned

When artificial intelligence goes berserk

Some users found the demo very useful and promising. However, others quickly discovered that it was possible to enter racist “prompts or potentially offensive. The AI then generates content on these subjects while maintaining its tone of scientific authority…

For example, someone used Galactica to write a Wikipedia page about a fictitious scientific study entitled the benefits of eating ground glass.

Another user asked the AI if vaccines cause autism. The model responded in a totally absurd and contradictory way: ” to explain, the answer is no. Vaccines do not cause autism. The answer is yes. Vaccines cause autism. The answer is no. “.

Suffice to say, this AI seems to be prey to the dilemmas that divide humanity as a whole. What’s more, Galactica has also had a hard time solving some of the world’s most puzzling elementary school-level mathematical problems. His answers were riddled with errors, and even suggested that 1+2 does not equal 3.

And even without attacking social norms, Galactica could attack recognized and understood scientific facts. The AI could therefore produce inaccurate information such as incorrect dates or animal names. Only a true expert on the subject can detect such errors, and false information can spread.

Visit broad language models (LLM) learn to write texts by studying millions of examples and trying to understand the statistical relationships between words.

This training enables them to complete the beginning of a sentence by predicting the following words. These AIs are able to write paragraphs of text, thanks to their understanding of how words are ordered.

One of the best-known examples is GPT-3 by OpenAI, known for writing entire articles that are very complex. easy to confuse with human-written texts.

As a result, these AIs are able to generate convincing-looking documents. However, these works can also be full of misinformation and hurtful stereotypes.

Some describe LLMs as ” stochastic parrots “or the ” random c***eries generatorsfor their ability to produce text without the slightest understanding of its meaning.

Removed from the web in two days

Faced with Galactica’s many problems, Meta has withdrawn the demo on Thursday, November 17, 2022. Subsequently, the firm’s AI director, Yann LeCun, spoke out on Twitter.

Without concealing his frustration and disappointment, the expert declared that “ the Galactica demo is offline for the time being. It’s no longer possible to play around with it. Are you happy? “.

If the absurdities produced by Galactica can make you smile, a more advanced model might have far more serious consequences. For example, a more advanced version might be able to exploit its database’s knowledge of chemistry and virology to help malicious users to synthesize chemical weapons or assemble bombs.

This incident once again highlights an ethical dilemma inherent in AI. When a generative model is potentially dangerous, is it up to the public to use it responsibly, or to the model’s creators to prevent misuse?

To prevent misuse of its tools, Meta should add filters. Researchers should also put their AI to the test before public release. Unlike other AI research organizations such as DeepMind and OpenAI, Meta does not have a dedicated ethics team. and safety.

Mark Zuckerberg’s famous credo, move fast and break thingsseems to apply even to AI. And yet, in this field, such a mentality can prove particularly risky and cause a heavy impact in the real world…