Open Access Open Access  Restricted Access Subscription Access

A MULTI-STREAM MODEL FOR DEEPFAKE FACE DETECTION USING SPATIAL, FREQUENCY AND TEMPORAL ANALYSIS

T. Sai Varun, P. Sushma, B. Lakshmi Prasanna, D. Aniram

Abstract


This project presents a multi-stream deepfake detection system designed to identify manipulated facial videos. To counter advanced generation methods, the architecture combines three analytical domains. Spatial features are extracted via a ResNet18 network, while hidden generative artifacts are uncovered using Fast Fourier Transform (FFT) for frequency-domain analysis. Furthermore, a Bi-LSTM network captures temporal inconsistencies across sequential video frames. These diverse features are integrated through an attention-based fusion module to maximize classification robustness. Validated on a processed video dataset, the system achieved a test accuracy of 94.21%, demonstrating significantly improved reliability over conventional single-stream detection approaches

Full Text:

PDF

References


A. Rössler, D. Cozzolino, L. Verdoliva, C. Riess, J. Thies, and M. Nießner, "FaceForensics++: Learning to Detect Manipulated Facial Images," Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 1-11, 2019.

I. Tolosana, R. Vera-Rodriguez, J. Fierrez, A. Morales, and J. Ortega-Garcia, "DeepFakes and Beyond: A Survey of Face Manipulation and Fake Detection," Information Fusion, vol. 64, pp. 131-148, 2020.

H. Dang, F. Liu, J. Stehouwer, X. Liu, and A. Jain, "On the Detection of Digital Face Manipulation," Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5781-5790, 2020.

D. Güera and E. J. Delp, "Deepfake Video Detection Using Intranuclear Sequence Analysis and CNN-LSTM," 15th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS), pp. 1-6, 2018.

E. Sabir, J. Cheng, A. Jaiswal, K. Rawls, P. Natarajan, and P. AbdAlmageed, "Recurrent Convolutional Strategies for Face Manipulation Detection in Videos," Interfaces (GUI), vol. 3, pp. 80-87, 2019.

K. He, X. Zhang, S. Ren, and J. Sun, "Deep Residual Learning for Image Recognition," Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 770-778, 2016.

J. Frank, T. Eisenhofer, L. Schönherr, A. Fischer, D. Kolossa, and T. Holz, "Leveraging Frequency Analysis for Deep Fake Image Recognition," International Conference on Machine Learning (ICML), vol. 119, pp. 3247-3258, 2020.

K. Zhang, Z. Zhang, L. Li, and Y. Qiao, "Joint Face Detection and Alignment Using Multitask Cascaded Convolutional Networks," IEEE Signal Processing Letters, vol. 23, no. 10, pp. 1499-1503, 2016.

T.-Y. Lin, P. Goyal, R. Girshick, K. He, and P. Dollár, "Focal Loss for Dense Object Detection," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 42, no. 2, pp. 318-327, 2017.

A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin, "Attention Is All You Need," Advances in Neural Information Processing Systems (NeurIPS), vol. 30, pp. 5998-6008, 2017.


Refbacks

  • There are currently no refbacks.