AI Regulation in the Generative AI Era
Recent AI regulatory initiatives are a crucial response to an urgent need, but raise questions about the role of governments in managing the disinformation crisis in the era of Generative AI.
More often than I care to admit, I get pulled into activities that contribute to the shaping of regulation or compliance standards for emerging technologies. Not because I fancy that, but because my team needs it. We apply regulatory compliance to everything we do, and if a framework is missing, we just can’t ship.
Sometimes it involves being the first to adopt a new compliance framework, other times it could be helping to answer regulators’ questions in the context of responsible use of AI in healthcare. Once I even found myself authoring the draft proposal for my country’s medical data de-identification standards, the local adaptation of the US Safe Harbor, together with my friend Roy Cohen from the Ministry of Health.
No complaints here. In the era of Generative AI, this area has become both critical and fascinating. It is impressive that regulators engage with tech experts for emerging technologies - things are moving so very fast these days, and it’s a proactive way to keep up with the rapid pace of technological advancement.
Just to be clear: I am not a regulator, not a lawyer, not a doctor. I am a computer scientist. I lead a multi-disciplinary R&D organization of over a hundred talented individuals, spread across several geographies around the world. We often push the envelope and apply new AI technologies to healthcare. And it is not uncommon that we are among the first to implement certain technologies.
This is why the AI-related regulation initiatives coming out in the last month got me interested: The Biden Executive Order on AI from US, followed by a hold-my-beer act from the European Parliament, aka EU AI Act. The UN also jumped on the bandwagon, issuing a report yesterday from the Secretary-General's Advisory Body on AI that has no enforcement authority, calling to strengthen international governance of AI.
The Biden Executive Order on AI
At the end of October 2023, the Biden administration issued an Executive Order on Safe, Secure, and Trustworthy Artificial Intelligence.
In a nutshell, Biden has issued an Executive Order aimed at ensuring the US leads in both leveraging the benefits and managing the risks of AI. The order aims to offer a comprehensive approach to responsibly manage the rapid advancement of AI technology. The key points of the order are around new standards for AI safety and security, promoting privacy, equity, civil rights, consumer and worker protection, innovation, competition, and global leadership.
This Executive Order is coming after the US Executive Order on Cybersecurity that was released on May 2021. One of the immediate actionable outcomes of that 2021 order was that every AI model needs to release information that includes what underlying components it is using and what data was used for training, to prevent poisoning of the data that will harm the model. Good and actionable.
Some interesting aspects of the new Biden AI Executive Order are in stating new safety standards:
The government will set standards for extensive red-team testing to ensure AI safety before public release, and companies will need to share the results of all red-team safety tests. I’ll share more on the joy of red-teaming in one of my next blog posts.
The government will also establish an advanced cybersecurity program to develop AI tools to find and fix vulnerabilities in critical software. The order also discusses combating algorithmic discrimination and advancing the responsible use of AI in healthcare and the development of affordable and life-saving drugs.
In the context of healthcare, The HHS will establish a safety program to handle reports of harms or unsafe healthcare practices involving AI.
One interesting area to highlight is this one: the order aims to protect Americans from AI-enabled fraud and deception by establishing standards for detecting AI-generated content and authenticating official content. As part of that, the government will develop guidance for content authentication and watermarking of AI-generated content to make it easy for Americans to know that the communications they receive from their government are authentic, “as an inspiration to the public sector”.
We will get back to watermarking in a second.
The EU AI Act
Few weeks after that, early December 2023, the European Union parliament announced the agreement on a comprehensive regulation for artificial intelligence, the AI Act.
In a nutshell, the EU approach classifies AI systems based on their associated risks and sets different requirements and obligations for each category. Some AI systems presenting ‘unacceptable’ risks would be prohibited, while ‘high-risk’ AI systems would be authorized but subject to a set of requirements and obligations.
While the AI Act is still not finalized, some interesting aspects can be highlighted:
The new law bans controversial "social scoring" systems — efforts to evaluate the compliance or trustworthiness of citizens. It restricts facial recognition technology in law enforcement to a handful of acceptable uses, including identification of victims of terrorism, human trafficking, and kidnapping.
The application of AI to healthcare is seen as highly beneficial. However, it is inherently categorized as a high-risk AI system. This categorization imposes additional conformity requirements and places event more stringent obligations on Generative AI.
Foundation model providers will need to submit detailed summaries of the training data they used to build their Generative AI models. Seems similar to the US Executive Order on Cybersecurity from 2021.
The new law will also require that operators of systems that are creating manipulated media will have to disclose that to users. Note this nuance: systems that are creating manipulated media. Not applications that are just surfacing that media.
The UN AI Advisory Body Report
Lastly, on December 21, the UN published an interim report from the Secretary-General's Advisory Body, discussing opportunities and risks of AI, and calling to strengthen international governance of AI. The report raises the societal challenges and has a lot of high-level need-to’s and should-have’s, but without concrete solutions yet, and with no practical tools to enforce regulation.
So, regulators are warming up. Hopefully, you too are starting to see the gaps.
AI holds many challenges, of different kinds. I will not even try to cover them all here. I want to focus on one aspect: disinformation.
Before we go there, some more background.
Watermarking
Watermarking of AI-generated content is a technique used to identify and authenticate content that has been produced by AI. This is done by embedding a signal or pattern into the content, which can be used to verify its origin. The primary purpose of watermarking is to help users distinguish between real and AI-generated content. Watermarking can be used to protect copyright or limit abuse of AI-generated content.
Whether the content is sound, images or video, Watermarking can be done by adding a visible or an invisible overlay onto the content or adding information into their metadata. For example, for AI-generated images, invisible watermarks are created by subtly modifying some pixels in the image, creating a pattern that is invisible to the human eye but can be detected by an AI system. Similarly, Video watermarking can be done by embedding a unique identifier into a video to help identify its ownership, copyright, and/or authenticity. Watermarking of sound files involves embedding a unique identifier into the audio signal by modifying the amplitude of the audio signal, typically using a complex algorithm to create a watermark that is difficult to remove without affecting the quality of the original audio. You get the drill.
However, while watermarking is a step forward, by itself the watermark is not likely to help end-users recognize fake or manipulated content, because it is mostly invisible.
And here comes the elephant in the room that both the US AI Executive Order and the EU AI Act are missing: the role of social media platforms in this, and the urgent need to enforce on the social media applications to surface the watermarks in the form of disclaimer on the posted content. It is not sufficient to enforce it on AI systems that generate the content - what good is watermarking for, if the end-user can’t see it?
The Era of Disinformation
As I mentioned in my previous blog post, fake news and disinformation are the pandemic of our era, tearing societies, and creating bias in public opinion. It seems that we have recently reached rock-bottom in this context, as the surge of misinformation is being exploited by harmful entities to sway public opinion, turning it into a tool of manipulation against nations.
It is not sufficient to enforce watermarking on AI systems that generate the content - what good is watermarking for, if the end-user can’t see it?
For a start, regulators should enforce on social media platforms the process of running a Watermarking detection on any piece of content that is uploaded by end-users. If a watermark exists, the social media platform should automatically add a visible, bold, non-removable disclaimer to the post that the content is generated by AI.
Disclaimers can be associated with AI-generated content regardless of the media type. And social media platforms are already performing some scanning. For example, just like YouTube scans videos and refuses to associate a soundtrack with a video if it has copyrights, it can put a bold disclaimer on a video if the content or its soundtrack are watermarked as AI-generated.
What about real footage that has been manipulated by users, generated prior to the watermarking act, or generated by malicious actors that do not adhere to the US or EU law? There’s still plenty that can be done, and social media platforms could even use AI for that. Let’s look at a couple of examples of disinformation - all real examples that were recently observed in social media:
AI-generated images or videos that have no watermark – use AI for image classification, followed by image fact-checking: for example, if there’s a cat in the picture, but it has 5 legs, then it is likely an AI-generated picture. AI generated images have flaws that can be identified.
Authentic media that is taken out of context – for example, a picture or video from an event that took place somewhere else in the world 5 years ago is presented as if it was taken recently someplace else – look at the media metadata for date and location, extract context from the associating text using natural language processing and cross-check the metadata and context. If they do not match the extracted from the text, it should be suspected as disinformation.
Text generated by bots can be detected by checking for text quality and consistency and can be cross-referenced with fact-checking systems for disinformation.
True that this is not perfect, and there are many loopholes still, but those techniques and many others would make spreading of disinformation somewhat harder.
The Role of Governments
Reportedly, the European Commission did reach out to tech platforms, asking them to detect photos, videos and text generated by artificial intelligence and clearly label them for users to help tackle disinformation, and allegedly Instagram has started working on it. But this should not be left to the discretion of the social media platform. It needs to be enforced.
The US and the EU should take a more assertive and proactive role in enforcing AI regulation, rather than merely having the public sector serve as an inspiration or pleading social media platform companies to behave responsibly - come on people, you are the regulators. Put your foot down.
The US Federal Government should enforce AI detection as a hard requirement on social media platform companies. Among those, companies like X, Meta and Google are American companies that must adhere to the law. The EU can do its part by implementing sanctions on social media platform companies that don’t detect and surface that content is AI-generated to end-users.
So, what about the Chinese-owned TikTok? Don’t get me started. Recently, it seems people have collectively lost their attention span due to this app, to the point that people are uncapable of comprehending a complex message that surpasses 15 seconds. Super shallow communication.
Well, the interesting thing is that TikTok is an app. In an app store. And the app stores are managed by Google and Apple. Both American companies. This means that the law enforced by US and EU could include the sanction of enforcing the removal of an app from the app store for that market if the app does not meet the law set by country’s regulator. That simple.
US and EU should take a more assertive and proactive role in enforcing AI regulation, rather than pleading social media platform companies to behave responsibly. You are the regulators. Put your foot down.
Handling disinformation is just one troubling aspect related to AI-generated content. There’s so much more. There are a lot of open-source models out there, making it harder to enforce regulation, especially in AI work coming out of entities that do not adhere to the US, EU, or international law. The US AI Executive Order acknowledges that this will not fly without international collaboration and includes a list of ally countries collaborating in this area for global impact. The UN is making its first attempts too. It’s not going to be easy, but it is our responsibility to make this right.
Welcome to the Verge of Singularity
Verge of Singularity is a blog that discusses Artificial Intelligence technology, its application in health & life sciences and the implications of emerging AI technologies on health tech and society. Posts are based on personal experiences. Opinions are my own.