Since their emergence a few years ago, artificial intelligence (AI)-synthesized media-so-called deep fakes-have dramatically increased in quality, sophistication, and ease of generation. Deep fakes have been weaponized for use in nonconsensual pornography, large-scale fraud, and disinformation campaigns. Of particular concern is how deep fakes will be weaponized against world leaders during election cycles or times of armed conflict. We describe an identity-based approach for protecting world leaders from deep-fake imposters. Trained on several hours of authentic video, this approach captures distinct facial, gestural, and vocal mannerisms that we show can distinguish a world leader from an impersonator or deep-fake imposter.

Department of Electrical and Computer Sciences School of Information University of California Berkeley CA 94708

Gymnasium of Johannes Kepler 169 00 Prague Czech Republic

See more in PubMed

Allyn B. Deepfake video of Zelenskyy could be ‘tip of the iceberg’ in info war, experts warn, https://www.npr.org/2022/03/16/1087062648/deepfake-video-zelenskyy-experts-war-manipulation-ukraine-russia (2022).

Chesney B., Citron D., Deep fakes: A looming challenge for privacy, democracy, and national security. Calif. Law Rev. 107, 1753 (2019).

Horton J., Sardarizadeh S.. False claims of ‘deepfake’ President Biden go viral, https://www.bbc.com/news/62338593 (2022).

Groh M., Epstein Z., Firestone C., Picard R., Deepfake detection by human crowds, machines, and machine-informed crowds. Proc. Natl. Acad. Sci. U.S.A. 119, e2110013119 (2022). PubMed PMC

Zhou P., Han X., Morariu V. I., Davis L. S., Two-stream neural networks for tampered face detection., International Conference on Computer Vision and Pattern Recognition (2017).

Li L. et al., Face X-ray for more general face forgery detection. http://arxiv.org/abs/1912.13458 (2019).

Li Y. et al., In ictu oculi: Exposing AI created fake videos by detecting eye blinking., International Workshop on Information Forensics and Security pp. 1–7. (2018).

Agarwal S., Farid H., Fried O., Agrawala M., Detecting deep-fake videos from phoneme-viseme mismatches in CVPR Workshop on Media Forensics. pp. 660–661. (2020).

Agarwal S. et al., Protecting world leaders against deep fakes in CVPR Workshop on Media Forensics. 1. (2019).

Cozzolino D., Rössler A., Thies J., Nießner M., Verdoliva L., ID-reveal: Identity-aware deepfake video detection in International Conference on Computer Vision. pp. 15108–15117. (2021).

Rössler A. et al., FaceForensics++: Learning to detect manipulated facial images in International Conference on Computer Vision and Pattern Recognition. (2019).

Baltrusaitis T., Zadeh A., Lim Y. C., Morency L. P., Openface 2.0: Facial behavior analysis toolkit in IEEE International Conference on Automatic Face& Gesture Recognition. pp. 59–66. (2018).

Gross R., Matthews I., Cohn J., Kanade T., Baker S., Multi-pie in IEEE International Conference on Automatic Face& Gesture Recognition. pp. 1–8. (2008). PubMed PMC

Ekman P., Friesen W. V., Measuring facial movement. Environ. Psychol. Nonverbal Behav. 1, 56–75 (1976).

Bazarevsky V. et al., Blazepose: On-device real-time body pose tracking. http://arxiv.org/abs/2006.10204 (2020).

Lugaresi C. et al., Mediapipe: A framework for building perception pipelines. http://arxiv.org/abs/1906.08172 (2019).

Boháček M., Hrúz M., Sign pose-based transformer for word-level sign language recognition in IEEE/CVF Winter Conference on Applications of Computer Vision Workshops. pp. 182–191. (2022).

Bauer A., The Use of Signing Space in a Shared Sign Language of Australia (De Gruyter, ed. 1, 2014).

Davis S., Mermelstein P., Comparison of parametric representations for monosyllabic word recognition in continuously spoken sentences. IEEE Trans. Acoust. Speech Signal Process. 28, 357–366 (1980).

Pedregosa F. et al., Scikit-learn: Machine learning in Python. J. Mach. Learn. Res. 12, 2825–2830 (2011).