Memento logo Deepfake Caricatures

Using Artifact Amplification to Expose Doctoring

Deepfakes are posing a serious threat to the integrity of our digital society. Detection techniques are essential to combat the evolving abilities of deepfake generation meth- ods. Crucially, developing tools that allow humans to recognize doctored videos at first glance is key to avoid the spread of misinformation. Here, we introduce a new deepfake detection framework that intelligently distorts fake videos to amplify unnatural deepfake artifacts. Our arti- fact amplification module is semi-supervised by human data and creates interpretable attention maps highlighting arti- facts. These maps improve the performance of our classifier, and allow us to generate Deepfake Caricatures: a transfor- mation of the original deepfake video where artifacts and subtle unnatural movements are amplified to improve hu- man recognition. In short, the contributions of this work are two-fold: we show improved performance on several deepfake detection datasets with a model explicitly attend- ing to artifacts of fake videos, and we provide a pipeline to create our Deepfake Caricatures, a defense mechanism against deepfakes to help people better identify if a video has been doctored. Our approach, based on a mixture of human and artificial supervision, aims to further the devel- opment of countermeasures against fake visual content, and grants humans the ability to make their own judgment when presented with dubious visual media

Dataset icon Framework and Data

Human Artifact Maps

Our Framework utilises both machine and human supervision to learn to highlight unnatural deepfake artifacts. This has two advantages: it allows us to build better deepfake detectors, and it gives rise to Deepfake Caricatures, a new way of modifying videos to expose doctoring.

code icon Code

Deepfake Caricature code Image

Download the code for the Deepfake Caricatures project here.

Paper icon Paper

Paper thumbnail
  • Camilo Fosco, Alex Andonian, Allen Lee, Xi Wang, Aude Oliva
    Deepfake Caricatures: Using Artifact Amplification To Expose Doctoring.

Gallery icon Gallery

From right to left: Real Video, Deepfake, Caricature on Fake and Caricature on Real. Hover to play the videos.