Foundation AI Models Like DALL.E & GPT-3 Need Community Norms Before they Harm

July 5, 2022

3 mins read

The independent policy release is highly problematic precisely because a single actor can’t predict the risks

A chatbot responding to human queries is nothing new but when it is spinning inflammatory and hyped comments on public forums like Twitter, it not only comes as surprise but becomes an issue of concern. Jerome Pesenti, VP, of Meta AI, in the year 2020 itself gave hints about the adverse effects the models can generate. In his tweet, he mentioned how GPT-3 when prompted with words like Jews, women, and holocaust, it generated a few utterly insane tweets. Few models perpetuate inequalities in irrevocable ways by amplifying biases because of the number of iterations the ML cycle goes through. Not only that AI models can have a societal impact but generate a huge environmental footprint. A study published in 2020 by Massachusetts University, found that approximately 626, 000 pounds of carbon dioxide is generated to train a particular AI model. Only a few years ago, a machine learning model that the UK government adapted generated test scores for students who never sat for the exam.

A case of consensus:

Foundational models or the AI code-based models on which many applications are developed have become ubiquitous. Google Search, a commonly used application depends heavily on BERT. And lakhs of people are now using GPT-3 to generate around 5 billion words a day. Given the risks they carry because of the inherent biases they are designed with, it becomes quite important that their release is monitored, or rather put under community norms. A release primarily means giving access to data, code, and models to external sources. However, individual developers have different approaches to do so. EleutherAI, Meta, and the BigScience project led by Hugging Face vouch for open release whereas OpenAI supports staged release by providing only API access to academic researchers. Companies like Google and DeepMind for PaLM and Gopher respectively, have gone for paper release.

While the model release lacks a general consensus, the independent policy release is highly problematic precisely because a single actor can’t predict the risks arising out of the model. Even if the developer is cautious and considers the risks, it is highly unlikely that the model would work the same way a few years from now, in a strategically different environment. Hence, it would be safe to assume that when an AI model is scrutinized by a group of people, testing it against certain benchmarks, there would be ample scope for mitigating the risks. This arrangement also provides an economic leeway for collaboration and quicker implementation of safer models.

Benefits vs risks of individual release: Definitely there is a rift when it comes to the open production of scientific knowledge which exposes the deep vulnerabilities the models carry. While there is no straight jacket solution to this problem, we can address it to some extent by going through different dimensions of the release policy. It is definitely an unachievable task when we look at the problem in binary. There are different dimensions to the entire release framework as to what part of the project is released, who are the end-users, and when and how it should be released.

What is the way out?

An AI model comes with different assets such as code, papers, models, and data having their sort of impact with respect to risk of harm. While assets like paper, code, and data are considered indirect assets having lesser impact unless it holds the hints for reproduction. Direct assets like developer-mediated assets, API access, and access to model weights can be largely misused by the end-user for it not only allows for a thorough reading of the model but also facilitates its deployment even if the terms of use prohibit it. When it comes to timing, it is generally believed that the release should happen in stages. Factors like intrinsic and external conditions, how much time has elapsed since the model is developed, and if it has passed all the temporal safety standards, would determine how deep an impact, negative or positive, it can make. It is quite ironic that even the well-minded people who are at the helm of affairs cannot or rather do not identify the thin line that separates theory development and deployment. Given the nascent stage and shallow penetration into the general public, AI models for most of their lifecycles will be categorized as ‘under development’ models. Therefore, most companies find it convenient to release even prototypes without proper warning labels. In Europe, cities like Amsterdam and Helsinki have mandated AI registries to record how algorithms deliver services, while the EU has released tough draft rules to categorize risk associated with AI models.