Emerging forms of generative AI misuse, which aren’t overtly malicious, still raise ethical concerns. For example, new forms of political outreach are blurring the lines between authenticity and deception, such as government officials suddenly speaking a variety of voter-friendly languages without transparent disclosure that they’re using generative AI, and activists using the AI-generated voices of deceased victims to plead for gun reform.
While the study provides novel insights on emerging forms of misuse, it’s worth noting that this dataset is a limited sample of media reports. Media reports may prioritize sensational incidents, which in turn may skew the dataset towards particular types of misuse. Detecting or reporting cases of misuse may also be more challenging for those involved because generative AI systems are so novel. The dataset also doesn’t make a direct comparison between misuse of generative AI systems and traditional content creation and manipulation tactics, such as image editing or setting up ‘content farms’ to create large amounts of text, video, gifs, images and more. So far, anecdotal evidence suggests that traditional content manipulation tactics remain more prevalent.
Staying ahead of potential misuses
Our paper highlights opportunities to design initiatives that protect the public, such as advancing broad generative AI literacy campaigns, developing better interventions to protect the public from bad actors, or forewarning people and equipping them to spot and refute the manipulative strategies used in generative AI misuse.
This research helps our teams better safeguard our products by informing our development of safety initiatives. On YouTube, we now require creators to share when their work is meaningfully altered or synthetically generated, and seems realistic. Similarly, we updated our election advertising policies to require advertisers to disclose when their election ads include material that has been digitally altered or generated.
As we continue to expand our understanding of malicious uses of generative AI and make further technical advancements, we know it’s more important than ever to make sure our work isn’t happening in a silo. We recently joined the Content for Coalition Provenance and Authenticity (C2PA) as a steering committee member to help develop the technical standard and drive adoption of Content Credentials, which are tamper-resistant metadata that shows how content was made and edited over time.
In parallel, we’re also conducting research that advances existing red-teaming efforts, including improving best practices for testing the safety of large language models (LLMs), and developing pioneering tools to make AI-generated content easier to identify, such as SynthID, which is being integrated into a growing range of products.
In recent years, Jigsaw has conducted research with misinformation creators to understand the tools and tactics they use, developed prebunking videos to forewarn people of attempts to manipulate them, and shown that prebunking campaigns can improve misinformation resilience at scale. This work forms part of Jigsaw’s broader portfolio of information interventions to help people protect themselves online.
By proactively addressing potential misuses, we can foster responsible and ethical use of generative AI, while minimizing its risks. We hope these insights on the most common misuse tactics and strategies will help researchers, policymakers, industry trust and safety teams build safer, more responsible technologies and develop better measures to combat misuse.