.png){kind=logo}
Open source generative AI models are safe to use behind company firewalls, can be downloaded for free, and can be utilized at scale without incurring expensive API call fees. But one has to stay alert. There are still risks, some of which are unique to generation AI.
In the modern world, it seems like everyone can train an AI model. However, if you do not know how to code or if you do not have training data, at best, it depends on the README description of your preferred open source generative AI model and republishing them as different names.
The 150 foundation models released in 2023 include the following: For the entire list of the journals. More than half of them, namely 67% were open source.
When conducting a survey among authors of an artificial intelligence journal, an impressive study spearheaded by the Stanford University stated that just 17 out of 6945 papers that were made available to the public for a total of three years used an open source option.
There is also a vast amount of variations. Currently, Hugging Face is considering the licensing for Llamas. Although Meta officially identifies the Llamce, it keeps track of over 80,000 LLMs only for text production.
It features a leaderboard that makes it easy to quickly sort the models based on their performance against different benchmarks. And these models are getting better very quickly, even though they lag behind the large commercial models.
The variety of open source licenses available is already a complex enough terrain. Is a project safe for use in businesses or just in non-profit settings? Can you change it and share it? Is it safe to integrate into a code base that is proprietary? With the advent of gen AI, there are some new issues. First, under a very loose understanding of the phrase, there exist new license types that are only about open source generative AI.
A family of models as a bespoke commercial license that balances open access to the models with responsibility and protections in place to help address potential misuse, the Llama family of models is among the strongest open source generative AI LLMs available.
It is legal to use the models for any business purposes, and it is allowed for developers to make more Llama derivative kinds of models and distribute them in any way they wish; however, it is prohibited to use Llama outputs to enrich other LLMs apart from the Llama derivatives. Additionally, businesses must apply for a license, which Meta may or may not give, if their affiliates have more than 700 monthly users. Built with Llama 3 must be displayed prominently if they choose to use Llama 3.
In a similar vein, Apple recently made Open ELM available under the Apple Sample Code License which was created specifically for the purpose and solely grants copyright permissions excluding patent rights.
Although the code is available, neither Apple nor Meta utilize widely recognized open source licenses. In fact, Apple released more than just the code; they also provided the pre-training parameters, the training logs, model weights, and even the training data set. This raises the other issue on licensing of open source software or products or applications depending on the authors preference.
Desperately simplified, the matter with further detail could be printed as follows: Only that code familiar to traditional open source software. Because it's open source, you may check its functionality and check for potential flaws.
But Open Source Generative AI is not just programming. Also, there are the model weights, fine tuning, and training data. To identify potential biases and understand how a model functions, all of those factors are essential.
A model tailored by North Korean hackers may do a poor job of correctly recognizing malware, or it may perform poorly when trained on, say, a library of conspiracy theories about the flat earth. Do open source LLMs make all of that data available? Since there are no standards, it varies on the model, or even the particular release of the model.
Anand Rao, a Carnegie Mellon University professor of AI and former global lead for AI at PwC, said that they can sometimes release the code, but without the fine tuning, you can spend a lot of money to achieve comparable performance.
A lot of open source projects are do-it-yourself projects. Businesses can download the code, but to get everything to function, they will need to employ consultants or have in-house experience. This is one of the major issues with Open Source Generative AI. The reason for this is simple since nobody has years of experience with such a cutting-edge technology.
However, if a business is brand-new to AI, it remains developed without a proprietary system, or it must expand rapidly, it is better to begin with a sole-source platform, says Rao.
As per him, ‘Getting the open source copy entails professional skills in downloads!’. However, it might be appropriate to consider open source alternatives once a business has completed its proof of concept, put the model into production, and the bills start to mount.
The lack of industry expertise also creates another problem for the open source generative AI space. One of the key advantages of open source is many people look at the code and can spot programming errors, security vulnerabilities, and other weaknesses.
But this thousand-eyes approach to open source security only works if there are, in fact, a thousand eyes capable of understanding what they’re seeing.
LLMs are infamously prone to jailbreaking, in which a user provides an ingenious prompt that deceives the device into breaching its rules and maybe producing malware. Commercial projects are specifically initiated by large organizations with motivated suppliers that can recognize such voids in the project and address them before they become an issue.
Moreover, vendors can monitor any strange behavior from the users if they were able to view the prompts that the users sent to the public copies of the models.
Enterprise versions of the products, which operate in private settings and do not reveal vendor prompts for model improvement, are less likely to be purchased by malicious actors. It's possible that no one on the team monitors for indications of jailbreaking in an open source project.
Additionally, malevolent actors can test possible hacks by downloading these models for free and using them in their own contexts. Since they can see the system prompt that the model employs and any other safeguards that the model creators may have added, the bad guys also gain an advantage in their jailbreaking efforts.
To counter these threats, enterprises need to ensure that they have strong and well implemented security against them when deploying AI models into their systems. Another element of resiliency is often made up of high-security measures, constant performance evaluations, and even high-stake model tests.
Rao says, “Not trial and error, The attacker can then, for instance, provide training data and see how to trick the Model into categorizing pictures in an improper unulated manner or provide a peculiar response to what seems to be an innocent command.”
If an AI model is to embed a watermark into the output generated by the model for a given input, then the adversary may have to look into the code and figure out how that particular watermark was placed in the output data. To find the faults, the attackers could try to analyze the model or some extra code and tools which could have been used.
Elena Sugis, a senior data scientist and capabilities lead at Nortal Global Digital Transformation Consultancy, said that you can just bombard the infrastructure with requests so that the model won’t.
Sugis said, “If we try to attack how the model came up with the output, then it will disrupt the entire system which could be volatile for the enterprise if the output of the model is used somewhere as a part of some other system.”
Elena Sugis’ insights identify the key problems AI models experience in intricate systems, and how they may be exploited publicly for overloading options or stopping the output formation process. Thus, they jeopardize the credibility of their output and are rather a threat towards the worth of the Internet for society usage.
By doing so, organizations can mitigate risks, maintain operational continuity, and uphold the trustworthiness of AI-driven outputs essential for informed decision-making and operational efficiency within their broader technological ecosystems.
1. Why is overwhelming infrastructure a concern for AI models
Overwhelming the infrastructure with requests can lead to performance degradation or system failure, impacting the reliability of AI outputs and overall system functionality.
2. What does it mean to attack the way the model creates output
Attacking the output creation process of an AI model involves manipulating inputs or system interactions to produce incorrect or misleading outputs, which can have serious consequences for downstream processes.
3. How can disruptions to AI model outputs affect larger systems
Disruptions can propagate through interconnected systems, potentially causing operational inefficiencies, financial losses, or even compromising data integrity and security.
4. Why is it harmful when AI model outputs are part of a bigger system?
AI outputs often serve as critical inputs for decision-making or operational processes within larger systems. Any disruption or inaccuracies in these outputs can ripple through the entire system, affecting business operations and outcomes.
5. What measures can enterprises take to protect AI models in complex environments?
Enterprises should implement robust security protocols, monitor system performance closely, and conduct thorough testing and validation of AI models to mitigate risks of attacks and ensure reliable performance within larger systems.
