Tag Archives: Regulation

The EU AI Act

In previous articles, we discussed the dangers of AI and the need for AI regulations. On 13 March 2024, the European Parliament approved the “Proposal for a Regulation of the European Parliament and of the Council laying down harmonised rules on artificial intelligence (Artificial Intelligence Act) and amending certain Union legislative acts.” The act was a proposal by the EU Commission of 21 April 2021. The act is usually referred to by its short name of the Artificial Intelligence Act, or the EU AI Act. In this article we look at the following questions: what is the EU AI Act? What is the philosophy of the EU AI Act? We discuss the limited risk applications and the high-risk applications. Finally, we also look at the EU AI Act’s entry into force and the penalties.

What is the EU AI Act?

As the full title suggests, it is a regulation that lays down harmonised rules on artificial intelligence across the EU. Rather than focusing on specific applications, it deals with the risks that applications pose, and categorizes them accordingly. The Act imposes stringent requirements for high-risk categories to ensure safety and fundamental rights are upheld. The Act’s recent endorsement follows a political agreement reached in December 2023, reflecting the EU’s commitment to a balanced approach that fosters innovation while addressing ethical concerns.

Philosophy of the EU AI Act

The AI Act aims to provide AI developers and deployers with clear requirements and obligations regarding specific uses of AI. At the same time, the regulation seeks to reduce administrative and financial burdens for business, in particular for small and medium-sized enterprises (SMEs). The aim of the new rules is to foster trustworthy AI in Europe and beyond, by ensuring that AI systems respect fundamental rights, safety, and ethical principles and by addressing risks of very powerful and impactful AI models.

The AI Act ensures that Europeans can trust what AI has to offer. Because AI applications and frameworks can change rapidly, the EU chose to address the risks that applications pose. While most AI systems pose limited to no risk and can contribute to solving many societal challenges, certain AI systems create risks that must be addressed to avoid undesirable outcomes. The Act distinguishes four levels of risk:

  • Unacceptable risk: applications with unacceptable risk are never allowed. All AI systems considered a clear threat to the safety, livelihoods and rights of people will be banned, from social scoring by governments to toys using voice assistance that encourages dangerous behaviour.
  • High risk: to be allowed high risk applications must meet stringent requirements to ensure safety and fundamental rights are upheld. We have a look at those below.
  • Limited risk: applications are considered to pose limited risk when they lack transparency, and the users of the application may not know what their data are used for or what that usage implies. Limited risk applications can be allowed if they comply with specific transparency obligations.
  • Minimal or no risk: The AI Act allows the free use of minimal-risk and no risk AI. This includes applications such as AI-enabled video games or spam filters. The vast majority of AI systems currently used in the EU fall into this category.

Let us have a closer look at the limited and high-risk applications.

Limited Risk Applications

As mentioned, limited risk refers to the risks associated with a lack of transparency. The AI Act introduces specific obligations to ensure that humans are sufficiently informed when necessary. E.g., when using AI systems such as chatbots, humans should be made aware that they are interacting with a machine so they can make an informed decision to continue or step back. Providers will also have to ensure that AI-generated content is identifiable. Besides, AI-generated text published with the purpose to inform the public on matters of public interest must be labelled as artificially generated. This also applies to audio and video content constituting deep fakes.

High Risk Applications

Under the EU AI Act, high-risk AI systems are subject to strict regulatory obligations due to their potential impact on safety and fundamental rights.

What are high risk applications?

As mentioned, all AI systems considered a clear threat to the safety, livelihoods and rights of people are considered high-risk. These systems are classified into three main categories: a) those covered by EU harmonisation legislation, b) those that are safety components of certain products, and c) those listed as involving high-risk uses. Specifically, high-risk AI includes applications in critical infrastructure, such as traffic control and utilities management, biometric and emotion recognition systems. It also applies to AI used in education and employment for decision-making processes.

What are the requirements for high-risk applications?

High-risk AI systems under the EU AI Act are subject to a comprehensive set of requirements designed to ensure their safety, transparency, and compliance with EU standards. These systems must have a risk management system in place to assess and mitigate potential risks throughout the AI system’s lifecycle. Data governance and management practices are crucial to ensure the quality and integrity of the data used by the AI, including provisions for data protection and privacy. Providers must also create detailed technical documentation that covers all aspects of the AI system, from its design to deployment and maintenance.

Furthermore, high-risk AI systems require robust record-keeping mechanisms to trace the AI’s decision-making process. This is essential for accountability and auditing purposes. Transparency is another key requirement, necessitating clear and accessible information to be provided to users and ensuring they understand the AI system’s capabilities and limitations. Human oversight is mandated to ensure that AI systems do not operate autonomously without human intervention, particularly in critical decision-making processes. Lastly, these systems must demonstrate a high level of accuracy, robustness, and cybersecurity to prevent errors and protect against threats.

The EU AI Act’s entry into force

The act will enter into force two years after it was approved, i.e., on 13 March 2026. This gives member states the opportunity to implement compliant legislation. It also gives providers a two-year window to adapt to the regulation.

The European AI Office, established in February 2024 within the Commission, will oversee the AI Act’s enforcement and implementation with the member states.

Penalties

The EU AI Act enforces a tiered penalty system to ensure compliance with its regulations. For the most severe violations, particularly those involving prohibited AI systems, fines can reach up to €35 million or 7% of the company’s worldwide annual turnover, whichever is higher. Lesser offenses, such as providing incorrect, incomplete, or misleading information to authorities, may result in penalties up to €7.5 million or 1% of the total worldwide annual turnover. These fines are designed to be proportionate to the nature of the infringement and the size of the entity, reflecting the seriousness of non-compliance within the AI sector.

Conclusion

The EU AI Act represents a significant step in the regulation of artificial intelligence within the EU. It sets a precedent as the first comprehensive legal framework on AI worldwide. The Act mandates a high level of diligence, including risk assessment, data quality, transparency, human oversight, and accuracy for high-risk systems. Providers and deployers must also adhere to strict requirements regarding registration, quality management, monitoring, record-keeping, and incident reporting. This framework aims to ensure that AI systems operate safely, ethically, and transparently within the EU. Through these efforts, the European AI Office strives to position Europe as a leader in the ethical and sustainable development of AI technologies.

 

Sources:

 

The dangers of artificial intelligence

Artificial intelligence (AI) is a powerful technology that can bring many benefits to society. However, AI also poses significant risks and challenges that need to be addressed with caution and responsibility. In this article, we explore the questions, “What are the dangers of artificial intelligence?”, and “Does regulation offer a solution?”

The possible dangers of artificial intelligence have been making headlines lately. First, Elon Musk and several experts called for a pause in the development of AI. They were concerned that we could lose control over AI considering how much progress has been made recently. They expressed their worries that AI could pose a genuine risk to society. A second group of experts, however, replied that Musk and his companions were severely overstating the risks involved and labelled them “needlessly alarmist”. But then a third group again warned of the dangers of artificial intelligence. This third group included people like Geoffrey Hinton, who has been called the godfather of AI. They even explicitly stated that AI could lead to the extinction of humankind.

Since those three groups stated their views, many articles have been written about the dangers of AI. And the calls to regulate AI have become louder than ever before. (We published an article on initiatives to regulate AI in October 2022). Several countries have started taking initiatives.

What are the dangers of artificial intelligence?

So, what are the dangers of artificial intelligence? As with any powerful technology, it can be used for nefarious purposes. It can be weaponized and used for criminal purposes. But even the proper use of AI holds inherent risks and can lead to unwanted consequences. Let us have a closer look.

A lot of attention has already been paid in the media to the errors, misinformation, and hallucinations of artificial intelligence. Tools like ChatGPT are programmed to sound convincing, not to be accurate. It gets its information from the Internet, but the Internet contains a lot of information that is not correct. So, its answers will reflect this. Worse, because it is programmed to provide any answer if it can, it sometimes just makes things up. Such instances have been called hallucinations. In a lawsuit in the US, e.g., a lawyer had to admit that the precedents he had quoted did not exist and were fabricated by ChatGPT. (In a previous article on ChatGPT, we warned that any legal feedback it gives must be double-checked).

As soon as ChatGPT became available, cybercriminals started using it to their advantage. A second set of dangers therefore has to do with cybercrime and cybersecurity threats: AI can be exploited by malicious actors to launch sophisticated cyberattacks. This includes using AI algorithms to automate and enhance hacking techniques, identify vulnerabilities, and breach security systems. Phishing attacks have also become more sophisticated and harder to detect.

AI can also be used for cyber espionage and surveillance: AI can be employed for sophisticated cyber espionage activities, including intelligence gathering, surveillance, and intrusion into critical systems. Related to this is the risk of invasion of privacy and data manipulation. AI can collect and analyse massive amounts of personal data from various sources, such as social media, cameras, sensors, and biometrics. This can enable AI to infer sensitive information about people’s identities, preferences, behaviours, and emotions. AI can also use this data to track and monitor people’s movements, activities, and interactions. This can pose threats to human rights, such as freedom of expression, association, and assembly.

Increased usage of AI will also lead to the loss of jobs due to automation. AI can perform many tasks faster and cheaper than humans, which will lead to unemployment and inequality. An article on ZD Net estimates that AI could automate 300 million jobs. Approximately 28% of current jobs could be at risk.

There also is a risk of loss of control. As AI systems become more powerful, there is a risk that we will lose control over them. This could lead to AI systems making decisions that are harmful to humans, such as launching a nuclear attack or starting a war. This risk of the loss of control is a major concern about the weaponization of AI. As AI technology advances, there is a worry that it could be weaponized by state or non-state actors. Autonomous weapon systems equipped with AI could potentially make lethal decisions without human intervention, leading to significant ethical and humanitarian concerns.

We already mentioned errors, misinformation, and hallucinations. Those are involuntary side-effects of AI.  A related danger of AI is the deliberate manipulation and misinformation of society through algorithms. AI can generate realistic and persuasive content, such as deepfakes, fake news, and propaganda, that can influence people’s opinions and behaviours. AI can also exploit people’s psychological biases and preferences to manipulate their choices and actions, such as online shopping, voting, and dating.

Generative AI tends to use existing data as its basis for creating new content. But this can cause issues of infringement of intellectual property rights. (We briefly discussed this in our article on generative AI).

Another risk inherent to the fact that AI learns from large datasets is bias and discrimination. If this data contains biases, then AI can amplify and perpetuate them. This poses a significant danger in areas such as hiring practices, lending decisions, and law enforcement, where biased AI systems can lead to unfair outcomes. And if AI technologies are not accessible or affordable for all, they could exacerbate existing social and economic inequalities.

Related to this are ethical implications. As AI systems become more sophisticated, they may face ethical dilemmas, such as decisions involving human life or the prioritization of certain values. Think, e.g., of self-driving vehicles when an accident cannot be avoided: do you sacrifice the driver if it means saving more lives? It is crucial to establish ethical frameworks and guidelines for the development and deployment of AI technologies. Encouraging interdisciplinary collaboration among experts in technology, ethics, and philosophy can help navigate these complex ethical challenges.

At present, there is insufficient regulation regarding the accountability and transparency of AI. As AI becomes increasingly autonomous, accountability and transparency become essential to address the potential unintended consequences of AI. In a previous article on robot law, we asked the question who is accountable when, e.g., a robot causes an accident. Is it the manufacturer, the owner, or – as AI becomes more and more self-aware – could it be the robot? Similarly, when ChatGPT provides false information, who is liable? In the US, Georgia radio host Mark Walters found that ChatGPT was spreading false information about him, accusing him of embezzling money. So, he is suing OpenAI, the creators of ChatGPT.

As the abovementioned example of the lawyer quoting non-existing precedents illustrated, there also is a risk of dependence and overreliance: Relying too heavily on AI systems without proper understanding or human oversight can lead to errors, system failures, or the loss of critical skills and knowledge.

Finally, there is the matter of superintelligence that several experts warn about. They claim that the development of highly autonomous AI systems with superintelligence surpassing human capabilities poses a potential existential risk. The ability of such systems to rapidly self-improve and make decisions beyond human comprehension raises concerns about control and ethical implications. Managing this risk requires ongoing interdisciplinary research, collaboration, and open dialogue among experts, policymakers, and society at large. On the other hand, one expert said that it is baseless to automatically assume that superintelligent AI will become destructive, just because it could. Still, the EU initiative includes the requirement of building in a compulsory kill switch that allows to switch the AI off at any given moment.

Does regulation offer a solution?

In recent weeks, several countries have announced initiatives to regulate AI. The EU already had its own initiative. At the end of May, its tech chief Margrethe Vestager said she believed a draft voluntary code of conduct for generative AI could be drawn up “within the next weeks”, with a final proposal for industry to sign up “very, very soon”. The US, Australia, and Singapore also have submitted proposals to regulate AI.

Several of the abovementioned dangers can be addressed through regulation. Let us go over some examples.

Regulations for cybercrime and cybersecurity should emphasize strong cybersecurity measures, encryption standards, and continuous monitoring for AI-driven threats.

To counter cyber espionage and surveillance risks, we need robust cybersecurity practices, advanced threat detection tech, and global cooperation to share intelligence and establish norms against cyber espionage.

Privacy and data protection regulations should enforce strict standards, incentivize secure protocols, and impose severe penalties for breaches, safeguarding individuals and businesses from AI-enabled cybercrime.

To prevent the loss of jobs, societies need to invest in education and training for workers to adapt to the changing labour market and create new opportunities for human-AI collaboration.

Addressing AI weaponization requires international cooperation, open discussions, and establishing norms, treaties, or agreements to prevent uncontrolled development and use of AI in military applications.

To combat deepfakes and propaganda, we must develop ethical standards and regulations for AI content creation and dissemination. Additionally, educating people on critical evaluation and information verification is essential.

Addressing bias and discrimination involves ensuring diverse and representative training data, rigorous bias testing, and transparent processes for auditing and correcting AI systems. Ethical guidelines and regulations should promote fairness, accountability, and inclusivity.

When it comes to accountability and transparency, regulatory frameworks can demand that developers and organizations provide clear explanations of how AI systems make decisions. This enables better understanding, identification of potential biases or errors, and the ability to rectify any unintended consequences.

At the same time, regulation also has its limitations. While it is important, e.g., to regulate things like cybercrime or the weaponization of AI, it is also clear that the regulation will not put an end to these practices. After all, by definition, cybercriminals don’t tend to care about any regulations. And despite the fact that several types of weapons of mass destruction have been outlawed, it is also clear that they are still being produced and used by several actors. But regulation does help to keep the trespassers accountable.

It is also difficult to assess how disruptive the impact of AI will be on society. Depending on how disruptive it is, additional measures may be needed.

Conclusion

We have reached a stage where AI has become so advanced that it will change the world and the way we live. This is already creating issues that need to be addressed. And as with any powerful technology, it can be abused. Those risks, too, need to be addressed. But while we must acknowledge these issues, it should also be clear that the benefits outweigh the risks, as long as we don’t get ahead of ourselves. At present, humans abusing AI are a greater danger than AI itself.

 

Sources:

 

Artificial Intelligence Regulation

In previous articles, we have discussed how artificial intelligence is biased, and on how this problem of biased artificial intelligence persists. As artificial intelligence (AI) is becoming ever more prevalent, this poses many ethical problems. The question was raised whether the industry could be trusted to self-regulate or whether legal frameworks would be necessary. In this article, we explore current initiatives for Artificial Intelligence regulation. We look at initiatives within the industry to regulate artificial intelligence as well as at attempts to create legal frameworks for Artificial Intelligence. But first we investigate why regulation is necessary.

Why is Artificial Intelligence Regulation necessary?

Last year, the Council of Europe published a paper where it concluded that a legal framework was needed because there were substantive and procedural gaps. UNESCO, too, identified key issues in its Recommendation on Ethics in Artificial Intelligence. Similarly, in its White Paper on Trustworthy AI, The Mozilla Foundation identifies a series of key challenges that need to be addressed and that makes regulation desirable. These are:

  • Monopoly and centralization: Large-scale AI requires a lot of resources and at present only a handful of tech giants have those. This has a stifling effect on innovation and competition.
  • Data privacy and governance:  Developing complex AI systems necessitates vast amounts of data. Many AI systems that are currently being developed by large tech companies harvest people’s personal data through invasive techniques, and often without their knowledge or explicit consent.
  • Bias and discrimination: As was discussed in previous articles, AI relies on computational models, data, and frameworks that reflect existing biases. This in turn results in biased or discriminatory outcomes.
  • Accountability and transparency: Many AI systems just present an outcome without being able to explain how that result was reached. This can be the product of the algorithms and machine learning techniques that are being used, or it may be by design to maintain corporate secrecy. Transparency is needed for accountability and to allow third-party validation.
  • Industry norms: Tech companies tend to build and deploy tech rapidly. As a result, many AI systems are embedded with values and assumptions that are not questioned in the development cycle.
  • Exploitation of workers: Research shows that tech workers who perform the invisible maintenance of AI are vulnerable to exploitation and overwork.
  • Exploitation of the environment: The amount of energy needed for AI data mining makes it very environment unfriendly. The development of large AI systems intensifies energy consumption and speeds up the extraction of natural resources.
  • Safety and security: Cybercriminals have embraced AI. They are able to carry out increasingly sophisticated attacks by exploiting AI systems.

For all these reasons, the regulation of AI is necessary. Many large tech companies still promote the idea that the industry should be allowed to regulate itself. Many countries, as well as the EU, on the other hand believe the time is ripe for governments to impose a legal framework to regulate AI.

Initiatives within the industry to regulate Artificial Intelligence

Firefox and the Mozilla Foundation

The Mozilla Foundation is one of the leaders in the field when it comes to promoting trustworthy AI. They already have launched several initiatives, including advocacy campaigns, responsible computer science challenges, research, funds, and fellowships. The Foundation also points out that “developing a trustworthy AI ecosystem will require a major shift in the norms that underpin our current computing environment and society. The changes we want to see are ambitious, but they are possible.” They are convinced that the “best way to make this happen is to work like a movement: collaborating with citizens, companies, technologists, governments, and organizations around the world.”

IBM

IBM, too, promotes an ethical and trustworthy AI, and has created its own ethics board. It believes AI should be built on the following principles:

  • The purpose of AI is to augment human intelligence
  • Data and insights belong to their creator
  • Technology must be transparent and explainable

To that end, it identified five pillars:

  • Explainability: Good design does not sacrifice transparency in creating a seamless experience.
  • Fairness: Properly calibrated, AI can assist humans in making fairer choices.
  • Robustness: As systems are employed to make crucial decisions, AI must be secure and robust.
  • Transparency: Transparency reinforces trust, and the best way to promote transparency is through disclosure.
  • Privacy: AI systems must prioritize and safeguard consumers’ privacy and data rights.

Google

Google says it “aspires to create technologies that solve important problems and help people in their daily lives. We are optimistic about the incredible potential for AI and other advanced technologies to empower people, widely benefit current and future generations, and work for the common good.

  1. Be socially beneficial
  2. Avoid creating or reinforcing unfair bias
  3. Be built and tested for safety
  4. Be accountable to people
  5. Incorporate privacy design principles
  6. Uphold high standards of scientific excellence
  7. Be made available for uses that accord with these principles.”

It also made it clear that it “will not design or deploy AI in the following application areas:

  1. Technologies that cause or are likely to cause overall harm. Where there is a material risk of harm, we will proceed only where we believe that the benefits substantially outweigh the risks and will incorporate appropriate safety constraints.
  2. Weapons or other technologies whose principal purpose or implementation is to cause or directly facilitate injury to people.
  3. Technologies that gather or use information for surveillance violating internationally accepted norms.
  4. Technologies whose purpose contravenes widely accepted principles of international law and human rights.”

It adds that that list may evolve.

Still, Google seems to have a troubled relationship with ethical AI. It notoriously fired its entire ethics board in 2019, to replace it with a team of ethical AI researchers. When subsequently, on separate occasions, two of those were fired too, it again made headlines.

Facebook / Meta

Whereas others talk about trustworthy and ethical Ai, Meta (the parent company of Facebook) on the other hand has different priorities and talks about responsible AI. It, too, identifies five (or ten) pillars:

  1. Privacy & Security
  2. Fairness & Inclusion
  3. Robustness & Safety
  4. Transparency & Control
  5. Accountability & Governance

Legal frameworks for Artificial Intelligence

Apart from those initiatives within the industry, there are proposals for legal frameworks as well. Best known is the EU AI Act. Others are following suit.

The EU AI Act

The EU describes its AI act as “a proposed European law on artificial intelligence (AI) – the first law on AI by a major regulator anywhere. The law assigns applications of AI to three risk categories. First, applications and systems that create an unacceptable risk, such as government-run social scoring of the type used in China, are banned. Second, high-risk applications, such as a CV-scanning tool that ranks job applicants, are subject to specific legal requirements. Lastly, applications not explicitly banned or listed as high-risk are largely left unregulated.”

The text can be misleading as, effectively, the proposal distinguishes not three but four levels of risk for AI applications: 1) unacceptable risk, which are banned, 2) high-risk, which must be regulated with specific legal requirements, 3) low risk, where most of the time no regulation will be necessary, and 4) no risk, which do not have to be regulated at all.

By including an ‘unacceptable risk‘ category, the proposal introduces the idea that certain types of AI applications should be forbidden because they violate basic human rights. All applications that manipulate human behaviour to deprive users of their free will, as well as systems that allow social scoring fall in this category. Exceptions are allowed for military purposes and law enforcement purposes.

High risk systems “include biometric identification, management of critical infrastructure (water, energy etc), AI systems intended for assignment in educational institutions or for human resources management, and AI applications for access to essential services (bank credits, public services, social benefits, justice, etc.), use for police missions as well as migration management and border control.” Again, there are exceptions, many of which have to do with cases where biometric identification is allowed. These include, e.g., missing children, suspects of terrorism, trafficking, and child pornography. The EU wants to create a database that keeps track of all high-risk applications.

Limited risk or low risk applications includes various bots which companies use to interact with their customers. The idea here is that transparency is required. Users must know, e.g., that they are interacting with a chat bot and to what information the chat bot has access.

All AI systems that do not pose any risk to citizen’s rights are considered no risk applications for which no regulation is necessary. These applications include games, spam filters, etc.

Who does the EU AI Act apply to? As is the case with the GDPR, the EU AI Act does not apply exclusively to EU-based organizations and citizens. It also applies to anybody outside of the EU who is offering an AI application (product or service) within the EU, or if an AI system uses information about EU citizens or organizations. Furthermore, it also applies to systems outside of the EU that use results that are generated by AI systems within the EU.

A work in progress: the EU AI Act is still very much a work in progress. The Commission made its proposal and now the legislators can give feedback. At present, more than a thousand amendments have been submitted. Some factions think the framework goes too far, while others claim it does not go far enough. Much of the discussions deal with both defining and categorizing AI systems.

Other noteworthy initiatives

Apart from the European AI Act, there are some other noteworthy initiatives.

Council of Europe: The Council of Europe (responsible for the European Convention on Human Rights) created its own Ad Hoc Committee on Artificial Intelligence. This Ad Hoc Committee published a paper in 2021, called A Legal Framework for AI Systems. The paper was a feasibility study explored the reasons as to why a legal framework on the development, design, and application of AI, based on Council of Europe’s standards on human rights, democracy and the rule of law is needed. It identified several substantive and procedural gaps and concluded that a comprehensive legal framework is needed, combining both binding and non-binding instruments.

UNESCO published a series of Recommendations on Ethics of Artificial Intelligence, which were endorsed by 193 countries in November 2021.

US: On 4 October, the White House released the Blueprint for an AI Bill of Rights to set up a framework that can protect people from the negative effects of AI.

No government initiatives exist yet in the UK. But Cambridge University, on 16 September 2022, published a paper on A Legal Framework for Artificial Intelligence Fairness Reporting.

 

Sources: