Is GenAI compliant with the AI Act, foundation models
currently, most genAI systems do not comply with the AI Act.
*LAST YEAR, A STUDY CONDUCTED BY THE CRFM (CENTER FOR RESEARCH ON FOUNDATION MODELS) AND THE HAI (HUMAN-CENTRED ARTIFICIAL INTELLIGENCE) CENTER AT STANFORD UNIVERSITY REVEALED HOW THE MAJORITY OF FOUNDATION MODELS (SUCH AS CHATGPT, LLAMA, OR CLAUDE) CURRENTLY DEPLOYED WORLDWIDE DO NOT COMPLY WITH THE EU AI ACT DRAFT REQUIREMENTS <BOMMASANI, KLYMAN, ET AL., 2023>
<AI Act>
Is GenAI compliant with the AI Act, foundation models
after nearly three years of work, on 13 march 2024, the European Parliament approved the AI Act, an initial regulatory framework marking the first step towards robust European (and global) AI legal regulation <'Brusselles effect', cfr. Wikipedia>
its main goal is to bring AI systems (not only genAI) in line with EU rights and values, by addressing legal and ethical issues such as security, privacy <Manheim, & Kaplan, 2019>, transparency <Larsson, & Heintz, 2020>, non-discrimination <Zuiderveen Borgesius, 2018>, social <Baldassarre, Caivano, et al., 2023> and environmental welfare <OECD, 2022>
specifically, the AI Act is characterised by a risk-based approach, which identifies four different levels of risk considering the potential societal implications of the AI systems.
unacceptable risk: AI systems under this category will be prohibited because of their dangerousness for users. this refers to systems manipulating people’s behaviour, or utilising social scoring to classify individuals according to personal characteristics, but also to those involving biometric identification in public spaces (although there is a limited scope of use for this latter sub-category in particularly severe cases).
it is remarkable to recall the efforts of the reclaim your face movement <https://reclaimyourface.eu/> in this regard, committed to fighting the use of facial recognition systems within public places.
*ART. 5 AI ACT
high risk: this involves all those AI systems that could hurt safety or fundamental rights. they are divided into two subgroups: AI systems embedded in products subject to EU product safety laws and AI systems operating in specific sectors (which must be registered within a specific database to ensure transparency on the part of providers). these include applications correlated to transport, health, and education (among others), such as automatic evaluation of students or AI applications in robot-assisted surgery.
*ART. 6 & SS. AI ACT
limited risk: systems in this category are required to comply with basic transparency measures to ensure users are aware they are interacting with an AI system and can, therefore, make responsible choices in their interactions. one of our aims should be to promote basic literacy in order to foster awareness among citizens, regarding AI models, their biases and limitations, and their potentially harmful consequences.
*ART. 52 AI ACT
*ART. 3(44BH) AI ACT
minimal risk: this includes AI systems already widely used in everyday life, such as spam filters, video games, and so on.
*ART. 69 AI ACT
<general purpose AI models>
Is GenAI compliant with the AI Act, foundation models
GPAI is the standard definition for foundation models within the AI Act, i.e. those artificial intelligence models trained on vast datasets to serve different purposes (applicable in various fields, through fine-tuning). these are, for instance, the genAI systems currently counting millions of users worldwide, such as ChatGPT (the chatbot based on the GPT-3.5 LLM, in the case of the free version, and on GPT-4, in the case of the paid version).
"GPAI model means an AI model, including when trained with a large amount of data using self-supervision at scale, that displays significant generality and is capable to competently perform a wide range of distinct tasks regardless of the way the model is placed on the market and that can be integrated into a variety of downstream systems or applications."
*ART. 3(44B) AI ACT
their broad scope of application, economic impact, and high pervasiveness led to an intricate debate on their regulation. indeed, genAI models are characterised by their ability to adapt to a wide variety of tasks without being specifically programmed for each one <Bommasani, Hudson, et al., 2021> and such flexibility introduces unique challenges in terms of security, privacy, and ethics. not only the products of these models are regulated (such as deepfakes*), but also the models themselves.
*MANIPULATED MULTIMEDIA CONTENT OR CONTENT GENERATED THROUGH THE USE OF GENERATIVE ARTIFICIAL INTELLIGENCE, RAISING CONCERNS ABOUT MISINFORMATION AND DEFAMATION.
a distinction is made between GPAI models with systemic and non-systemic risk, whereby the former have more stringent obligations.
"A general-purpose AI model shall be classified as general-purpose AI model with systemic risk if it meets any of the following criteria:
(a) it has high impact capabilities evaluated on the basis of appropriate technical tools and methodologies, including indicators and benchmarks;
(b) based on a decision of the Commission, ex officio or following a qualified alert by the scientific panel that a general purpose AI model has capabilities or impact equivalent to those of point a)."
*ART. 51(1) AI ACT
specifically, concerning point a, the computing power used for its training must be greater than 10²⁵ FLOPS (floating point operations per second), as in the case of Google’s PaLM. moreover, systemic risk can be assessed based on other factors, such as the complexity of the model’s parameters and its diffusion among consumers.
<obligations>
the AI Act introduces several obligations for providers of GPAI models in terms of transparency: as already mentioned, users must be informed that they are interacting with an AI, and outputs must be labelled in a machine-readable format.
in addition to this, providers shall keep up-to-date documentation regarding:
a general description of the GPAI model (e.g. tasks, input and output methodologies, parameters)
the design choices and training methodologies and techniques
the sources of the data used, and the bias mitigation techniques employed
information concerning computational resources and energy consumption must also be provided, responding to a growing concern about the huge environmental impact of these models on an already fragile planet.
some of these obligations, however, do not apply to those models that are open source and whose use, model modification, and parameters are made public.
*ART. 53 AI ACT
*ANNEX XI AI ACT
for providers of systemic risk GPAI models, there are additional obligations. firstly, the inclusion of the models in a public database (like high risk systems) and the timely reporting of any dangerous incidents. secondly, adopting strategies for risk assessments and thorough cybersecurity measures. lastly, conducting and documenting adversarial testing in order to identify and mitigate systemic risk.
the AI Office will be established within the European Commission to monitor the compliance of GPAI models within the EU.
<most genAI systems don't comply with the AI Act>
Is GenAI compliant with the AI Act, foundation models
the study conducted in 2023 (when the draft of the AI Act was not yet definitive) by Stanford University demonstrated how most artificial generative intelligence models, such as GPT-4, Stable Diffusion v2, and PaLM 2, do not comply with the requirements set by the EU.
in particular, two critical dimensions emerge as particularly problematic: the use of copyrighted training data – for which OpenAI was recently sued by the New York Times <Grynbaum, & Mac, 2023> – and the procedures adopted in the evaluation and testing of these models. also the environmental impact has been rarely addressed by genAI providers.
the study emphasised particularly a lack of transparency in the models’ development and dissemination processes. the results of the survey, reported in a detailed table (below), reveal that most providers not only fail to disclose the data sources that have been used, but also fail to implement effective strategies aimed at mitigating biases and assessing the societal and ethical impact of their products <Bommasani, Hudson, et al., 2021>. this issue raises significant concerns about the capacity of these genAI systems to cooperate safely and fairly with us.
Source: Bommasani, R., Klyman, K., Zhang, D., & Liang, P. (2023). Do foundation model providers comply with the eu AI Act. Stanford Center for Research on Foundation Models, Institute for Human-Centered Artificial Intelligence.
this table indicates that the path to full compliance is long and probably fraught with obstacles. nevertheless, the importance of this regulatory effort must not be underestimated. although not everyone agrees with the requirements laid down, the AI Act represents a first step towards integrating ethical and legal principles into genAI development, dissemination, and interaction, at a larger scale; while promoting a responsible and aware use of these technologies. moreover, by promoting transparency of information and replicability, the act aims to foster the development of open-source models <Gibney, 2024>
however, the European initiative received heavy criticism from some large companies such as Siemens and Airbus, which signed an open letter last year. their criticism was directed at too stringent regulation of foundation models, which would undermine the economic competitiveness of European companies compared to states with lighter policies. the demands were not taken into consideration.
we are now only a few weeks away from the AI Act publication in the Official Journal: 12 months after entry into force, the rules and obligations for GPAI will become applicable.
the approbation by the European Parliament marks a decisive step in the international policy regarding foundation models. through regulatory pressure and a growing awareness of ethical challenges also among users, providers of these models will hopefully take concrete actions in order to align with the required standards, thus ensuring a future in which cooperation with advanced genAI technologies is both ethical and beneficial to individuals, society as a whole, and natural environments.
Is GenAI compliant with the AI Act, foundation models
<references>
Baldassarre, M. T., Caivano, D., Fernandez Nieto, B., Gigante, D., & Ragone, A. (2023, September). The social impact of generative ai: An analysis on chatgpt. In Proceedings of the 2023 ACM Conference on Information Technology for Social Good (pp. 363-373).
Bommasani, R., Hudson, D. A., Adeli, E., Altman, R., Arora, S., von Arx, S., … & Liang, P. (2021). On the opportunities and risks of foundation models. arXiv preprint arXiv:2108.07258.
Bommasani, R., Klyman, K., Zhang, D., & Liang, P. (2023). Do foundation model providers comply with the eu ai act. Stanford Center for Research on Foundation Models, Institute for Human-Centered Artificial Intelligence.
Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J. D., Dhariwal, P., … & Amodei, D. (2020). Language models are few-shot learners. Advances in neural information processing systems, 33, 1877-1901.
Espinoza, J. (2023, June 30). European companies sound alarm over draft AI law. Financial Times, https://www.ft.com/content/9b72a5f4-a6d8-41aa-95b8-c75f0bc92465.
Grynbaum, M. M., & Mac, R. (2023, December 27). The Times sues OpenAI and Microsoft over A.I. use of copyrighted work. The New York Times. https://www.nytimes.com/2023/12/27/business/media/new-york-times-open-ai-microsoft-lawsuit.html.
Gibney, E. (2024). What the EU’s tough AI law means for research and ChatGPT. Nature.
Helberger, N., & Diakopoulos, N. (2023). ChatGPT and the AI Act. Internet Policy Review, 12(1).
Larsson, S., & Heintz, F. (2020). Transparency in artificial intelligence. Internet Policy Review, 9(2).
Manheim, K., & Kaplan, L. (2019). Artificial intelligence: Risks to privacy and democracy. Yale JL & Tech., 21, 106.
Zuiderveen Borgesius, F. (2018). Discrimination, artificial intelligence, and algorithmic decision-making. Council of Europe, Directorate General of Democracy, 42.
OECD (2022), “Measuring the environmental impacts of artificial intelligence compute and applications: The AI footprint”, OECD Digital Economy Papers, No. 341, OECD Publishing, Paris, https://doi.org/10.1787/7babf571-en.