To find a suitable opener for this blog article, I truly fell down the rabbit hole. In my search on Google for the latest deepfakes over the past 24 hours, I came across 21 pages of search results. I got stuck on one example, both because it’s too absurd to be true and because people were financially harmed.
In July 2024, YouTube was flooded with live streams from the Bitcoin 2024 conference (see Figure 1). The problem: the videos were neither live nor real – deepfakes showing Elon Musk claiming “he would personally double any cryptocurrency sent to his account”1. Although the statement is unbelievable, one specific video alone collected over $28,000 in crypto2. The community on Reddit is up in arms: “YouTube is full of ‘live’ scam shit especially now around the event of Bitcoin 2024 Nashville. Don’t fall for all of this […]. NOBODY IS GIVING AWAY MONEY FOR FREE. You will lose what is claimed to be sent back ‘doubled’. Just a friendly reminder! Don’t be stupid!”3.
Figure 1: Screenshot from YouTube, published by The New York Times4
Science & the economy
In science and the economy, misinformation and disinformation (including deepfakes) are now among the biggest challenges that not only occupy research and innovation teams but also force governments and companies to act. No wonder, as these dangers threaten to shake the stability of our democracies and erode trust in social institutions. Social media is a powerful amplifier. Its reach and speed make it the perfect channel for disinformation campaigns. In an era where artificial intelligence can create deceptively real content faster and cheaper than ever before, we face an unprecedented challenge. A worrying study by the World Economic Forum shows that 53% of surveyed risk experts rank AI-generated disinformation and misinformation as the second greatest risk for a global material crisis5.
Complex problems require complex solutions
There is no silver bullet that can solve all problems related to misinformation and manipulated media content. But we need to rebuild the trust in media content again and help the end user to make a trust decision based on as much and solid data as possible.
Experts in media and research agree: the key lies in multi-layered, comprehensive approaches to combat disinformation6. One example of a pioneering initiative is the Coalition for Content Provenance and Authenticity (C2PA)7. This group, including major companies like Google, Adobe, Microsoft, and Sony, is working on solutions to ensure that the authenticity and provenance of media content can be verified. All elements from the C2PA architecture are shown in Figure 2.
Figure 2: C2PA Architecture, as published in the C2PA Specification8
Their approach relies on so called C2PA manifests. A C2PA manifest is a cryptographically verifiable dataset that contains assertions about an asset as well as its associations. It enables actors, such as validators or consumers, to verify the authenticity and integrity of digital content by validating the signature and evaluating the assertions. This manifest ensures that the digital content remains traceable and trustworthy throughout its lifecycle.
Our experiment with C2PA
In the scope of our FAMIUM Provenance and Authentication Solution, we are also working with C2PA. To explore how C2PA enhances the protection and verification of digital content, we developed a demo using C2PA open-source tools to test how the standard performs in practice and examined its look and feel. The demo allows to authenticate videos against existing manifests and add new or additional manifests to video content.
Additionally, we tested the integration of the dash.js player with C2PA9 (explore our DASH activities). The integration validates the hashes of the DASH segments with the ones stated in the C2PA manifest . In case of mismatches, the user is informed that the content was tampered with. Further, the C2PA manifest is displayed to give the user information about the origin and author of the video. This is a promising step towards more trustworthy digital media and improved transparency for viewers and creators. Nevertheless, the goal has not been achieved yet.
Figure 3: Our demo using C2PA to explore technical solutions for content provenance
Challenges in the application in real-world scenarios:
While C2PA provides a standardised way to add metadata to different file types, it lacks guidance on how to apply the technology in modern real-world scenarios that go beyond concentrating on existing known and complete files. Questions arise on topics such as user acceptance and quality of experience, dynamic packaging, multi-CDN handling, live streaming and what to do if access to the original manifest data in a file is not possible.
Currently, we are working on a proof of concept for using C2PA manifests in live streaming context with dash.js and ffmpeg (see intermediate status in Figure 4). This involves signing, sending, and checking the individual segments against the C2PA manifest in real-time. The key difference is that the video content is sent as a live stream – piece by piece – and is not fully available beforehand. This means that a cryptographic hash cannot be calculated about the full content but needs to be updated in real-time. A cryptographic function is essential to authenticate manifest data and the related media. Authentication and provenance data must be calculated regularly, secured, sent to the client, and evaluated. These are costly and complex steps, but the aim is to implement a solution that does not affect the user’s quality of experience.
Figure 4: Interim status of the demo we are currently working on
Another aspect we are working on is the handling of metadata loss and end-user acceptance.
Playout Infrastructure:
The complex playout infrastructure with production and delivery infrastructures, including encoders and packagers, and even third parties like CDNs, presents multiple challenges. The question is how can it be ensured that C2PA manifests are preserved or enriched as the content is distributed through all intermediate steps until reaching the playback client? Typically, components that are not C2PA compliant loose the embedded information during a processing step. For example, just open an image that has embedded C2PA data in MS Paint and save the image again. The embedded data is lost.
Overcoming Barriers to Metadata Access:
What happens if media content is, for example, recorded from screens? Or just sent as broadcast transport streams via satellite? In those cases, the embedded C2PA data does not reach the end user. So how can metadata loss be restored in these cases? Methods such as watermarking or fingerprinting might help in this regard to identify media items. Using these methods, manifest data can be retrieved from additional services that are decoupled from the media itself.
Educating Users and Enhancing Trust in Media Authenticity
Finally, user acceptance of approaches like C2PA needs to be evaluated. How can the C2PA manifest be made understandable for the user? How can changes during live streams be signaled, where every second could essentially originate from a different source and may or may not be authentic? How can users be educated about the technology (“user education”)? What information does the user need to make an informed trust decision?
Lots of work to do around media provenance and fighting fake news and misinformation. We’ll keep you updated. In the meantime, check out our website for FAMIUM solutions, such as Content Provenance and DASH.
- https://www.nytimes.com/interactive/2024/08/14/technology/elon-musk-ai-deepfake-scam.html Zugriff am 11. November 2024 ↩︎
- https://news.bitcoin.com/de/betrueger-verwenden-deepfake-elon-musk-video-um-waehrend-des-bitcoin-2024-events-krypto-zu-stehlen/ Zugriff am 11. November 2024 ↩︎
- https://www.reddit.com/r/Bitcoin/comments/1ecpnwg/ youtube_is_full_of_live_scam_shit_especially_now/ Zugriff am 11. November 2024 ↩︎
- https://www.nytimes.com/interactive/2024/08/14/technology/elon-musk-ai-deepfake-scam.html Zugriff am 11. November 2024 ↩︎
- World Economic Forum. (10. Januar, 2024). Welche fünf Risiken werden Ihrer Meinung nach im Jahr 2024 am ehesten eine globale materielle Krise darstellen? [Graph]. In Statista. Zugriff am 01. November 2024, von https://de.statista.com/statistik/daten/studie/1458887/umfrage/die-groessten-akuten-risiken-fuer-wohlstand-und-frieden-auf-der-welt-des-jahres ↩︎
- By Jon Bateman and Dean Jackson Bateman, Jon; Jackson, Dean: “Countering Disinformation Effectively. Countering Disinformation Effectively: An Evidence-Based Policy Guide. 31. Januar 2024. URL: https://carnegieendowment.org/research/2024/01/countering-disinformation-effectively-an-evidence-based-policy-guide?lang=en ↩︎
- https://c2pa.org/ ↩︎
- https://c2pa.org/specifications/specifications/1.4/specs/C2PA_Specification.html ↩︎
- https://github.com/contentauth/dash.js/tree/c2pa-dash ↩︎