OpenAI and Meta have warned U.S. federal rulemakers that increasingly realistic AI-generated media could make it harder for judges and juries to determine whether digital evidence is genuine.
The warning came during an October meeting of the Judicial Conference’s Advisory Committee on Evidence Rules, which has been examining whether existing evidentiary rules are sufficient for audio, video, images and text that may have been generated or manipulated with AI.
Reuters reported that representatives from OpenAI and Meta told the committee that generative AI is making fabricated audio and video increasingly difficult to distinguish from authentic material.
Courts have not been flooded with deepfakes yet
The problem remains more anticipatory than widespread.
Reuters reported that only 15 of 931 judges surveyed had encountered challenges involving suspected deepfake evidence. But the committee has been studying the issue since 2023 as generative tools become cheaper and easier to use.
The committee’s official October agenda says that, so far, deepfakes have not flooded the courts, but asks whether increasing accessibility and decreasing detectability could raise that risk in the future.
That distinction matters. Rules written for a rare problem today may need to work in an environment where synthetic media can be produced at negligible cost tomorrow.
A new authentication rule is under discussion
Federal rulemakers have already drafted a possible Rule 901(c).
The committee’s agenda says the proposed rule would require a party challenging evidence to first provide evidence sufficient to support a finding that the item is a deepfake. If that threshold is met, the party introducing the evidence would then need to show that it is more likely than not authentic.
The committee has not finally approved that proposal.
OpenAI policy counsel Shira Anderson also warned that no foolproof method exists for identifying AI-generated content and that one is unlikely to emerge in the foreseeable future, Reuters reported.
Watermarking and provenance standards could help, but the judiciary’s own agenda specifically asks whether systems such as C2PA will be sufficient when people can create synthetic material using tools outside compliant platforms.
Digital evidence is becoming a product-design problem
U.S. Courts has previously said that AI-generated and AI-processed evidence raises fundamental questions regarding authentication, reliability, and the establishment of proper foundations.
For technology companies, that turns provenance into more than a moderation feature.
Cameras, editing tools, messaging platforms and generative systems may increasingly be expected to preserve information showing where media came from and what happened to it afterward.
The challenge is that authenticity infrastructure only works if enough of the ecosystem participates.
A watermark can identify content created by a compliant model. It cannot prove that unmarked material is authentic.
That is why deepfakes may ultimately create a larger technological shift: courts may need evidence systems that establish how digital media was created and handled, rather than relying only on whether software can spot a fake after the fact.
