Deepfake Video Detection Method Improved by GRU and Involution

doi:10.3778/j.issn.1002-8331.2206-0510

Abstract

Abstract: In recent years, the wide spread of deepfake video on the network has caused a negative impact. In order to solve the problems of low accuracy of existing detection models and insufficient and comprehensive information extraction, an improved deepfake video detection method based on gated recurrent unit（GRU） and Involution is proposed. Firstly, a feature extraction network is constructed based on Involution operator to extract global feature information, which enhances the spatial modeling ability of face image from spatial and channel information. Then, the temporal features are extracted through the location and inter-frame information of the main capsule layer and GRU concern features. Finally, focalloss is used as the loss function to balance the samples in the training model phase. The method is tested in Deepfakes, FaceSwap and Celeb-DF datasets, and the experimental results show that the method is better than the mainstream detection methods. The improved comparative experiments further prove the effectiveness of the detection method.

Key words: deepfake, gated recurrent unit（GRU）, Involution, capsule network, focalloss

摘要： 近年来深度伪造视频在网络上广泛传播，造成了负面影响。针对现有检测模型准确率低和信息提取不够充分和全面的问题，提出了一种GRU（gated recurrent unit）和Involution改进的深度伪造视频检测方法。首先使用VGG19作为主干网络提取空间特征，并将Involution算子嵌入主干网络，从空间和通道信息两方面加强了人脸图像的空间建模能力。然后通过主胶囊层关注特征的位置信息和使用GRU提取帧间的时序特征。最后在训练模型阶段使用focalloss作为损失函数来平衡样本。在Deepfakes、FaceSwap和Celeb-DF数据集中进行测试，实验结果表明该方法优于主流检测方法，改进对比实验进一步证明了检测方法的有效性。

关键词: 深度伪造, 门控循环单元（GRU）, Involution, 胶囊网络, focalloss

LIU Yalin, LU Tianliang. Deepfake Video Detection Method Improved by GRU and Involution[J]. Computer Engineering and Applications, 2023, 59(22): 276-283.

刘亚琳, 芦天亮. GRU和Involution改进的深度伪造视频检测方法[J]. 计算机工程与应用, 2023, 59(22): 276-283.

References

[1] STUPP C.Fraudsters used AI to mimic CEO’s voice in unusual cybercrime case[J].The Wall Street Journal，2019，30（8）.
[2] SUWAJANAKORN S，SEITZ S I，KEMELMACHER-SHLIZERMAN I.Synthesizing Obama：learning lip sync from，audio[J].ACM Transactions on Graphics，2017，36（4）：95.
[3] KOOPMAN M，RODRIGUEZ A M，GERADTS Z.Detection of deepfake video manipulation[C]//Proceedings of the 20th Irish Machine Vision and Image Processing Conference，2018：133-136.
[4] 李纪成，刘琲贝，胡永健，等.基于光照方向一致性的换脸视频检测[J].南京航空航天大学学报，2020，52（5）：760-767.
LI J C，LIU B B，HU Y J，et al.Deepfake video detection based on consistency of illumination direction[J].Journal of Nanjing University of Aeronautics & Astronautics，2020，52（5）：760-767.
[5] YANG J，XIAO S，LI A，et al.Detecting fake images by identifying potential texture difference[J].Future Generation Computer Systems，2021，125（7553）：127-135.
[6] DEMIR I，CIFTCI U A.Where do deep fakes look? Synthetic face detection via gaze tracking[C]//Proceedings of the 2021 ACM Symposium on Eye Tracking Research and Applications，2021：1-11.
[7] LI X，LANG Y，CHEN Y，et al.Sharp multiple instance learning for deepfake video detection[C]//Proceedings of the 28th ACM International Conference on Multimedia，2020：1864-1872.
[8] NGUYEN H，YAMAGISHI J，ECHIZEN I.Use of a capsule network to detect fake images and videos[J].arXiv：1910.12467，2019.
[9] HARA K，KATAOK H，SATOH Y.Learning spatio-temporal features with 3D residual networks for action recognition[C]//Proceedings of the 2017 IEEE International Conference on Computer Vision Workshops，Venice，2017：3154-3160.
[10] CARREIRA J，ZISSERMAN A.Quo vadis，action recognition? A new model and the kinetics dataset[C]//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition，Honolulu，2017：4724-4733.
[11] WANG Y，DANTCHEVA A.A video is worth more than 1000 lies.Comparing 3DCNN approaches for detecting deepfakes[C]//Proceedings of the 2020 15th IEEE International Conference on Automatic Face and Gesture Recognition，2020.
[12] SEBYAKIN A，SOLOVIEV V，ZOLOTARYUK A.Spatio-temporal deepfake detection with deep neural networks[C]//Proceedings of the 16th International Conference on Diversity，Divergence，Dialogue.Cham：Springer，2021：78-94.
[13] GANIYUSUFOGLU I，NG? L M，SAVOV N，et al.Spatio-temporal features for generalized detection of deepfake videos[J].arXiv：2010.11844，2020.
[14] XUAN H N，TRAN T S，LE V T，et al.Learning spatio-temporal features to detect manipulated facial videos created by the deepfake techniques[J].Forensic Science International Digital Investigation，2021，36（4）：301108.
[15] CALDELLI R，GALTERI L，AMERINI I，et al.Optical flow based CNN for detection of unlearnt deepfake manipulations[J].Pattern Recognition Letters，2021，146：31-37.
[16] HU J，LIAO X，WANG W，et al.Detecting compressed deepfake videos in social networks using frame-temporality two-stream convolutional network[J].IEEE Transactions on Circuits and Systems for Video Technology，2022，32（3）：1089-1102.
[17] BCA B，Tl A，WD C.Detecting deepfake videos based on spatiotemporal attention and convolutional LSTM[J].Information Sciences，2022，601：58-70.
[18] SABIR E，CHENG J，JAISWAL A，et al.Recurrent convo-lutional strategies for face manipulation detection in videos[J].arXiv：1905.00582，2019.
[19] GUERA D，DELP E J.Deepfake video detection using recurrent neural networks[C]//Proceedings of the 2018 15th IEEE International Conference on Advanced Video and Signal Based Surveillance，2018.
[20] XIAO Y，YIN H，ZHANG Y，et al.A dual-stage attention-based Conv-LSTM network for spatio-temporal correlation and multivariate time series prediction[J].International Journal of Intelligent Systems，2021，36（5）：2036-2057.
[21] MASI I，KILLEKAR A，MASCARENHAS R M，et al.Two-branch recurrent network for isolating deepfakes in videos[C]//Proceedings of the 16th European Conference on Computer Vision.Cham：Springer，2020：667-684.
[22] WU X，XIE Z，GAO Y T，et al.SSTNet：detecting manipulated faces through spatial，steganalysis and temporal features[C]//Proceedings of the 2020 IEEE International Conference on Acoustics，Speech and Signal Processing，2020：2952-2956.
[23] WANG Z，LI X，NI R，et al.Attention guided spatio-temporal artifacts extraction for deepfake detection[C]//Proceedings of the 4th Chinese Conference on Pattern Recognition and Computer Vision.Cham：Springer，2021：374-386.
[24] SU Y，XIA H，LIANG Q，et al.Exposing deepfake videos using attention based convolutional LSTM network[J].Neural Processing Letters，2021，53（6）：4159-4175.
[25] CHEN B，LI T，DING W.Detecting deepfake videos based on spatiotemporal attention and convolutional LSTM[J].Information Sciences，2022，601：58-70.
[26] KIMY，KOH Y J，LEE C，et al.Dark image enhancement based on pairwise target contrast and multiscale detail boosting[C]//Proceedings of the 2015 IEEE International Conference on Image Processing，2015：1404-1408.
[27] SIMONYAN K，ZISSERMAN A.Very deep convolutional networks for large-scale image recognition[J].arXiv：1409.
1556，2014.
[28] HINTON G E，KRIZHEVSKY A，WANG S D.Transforming auto-encoders[C]//Proceedings of the 21st International Conference on Artificial Neural Networks.Berlin，Heidelberg：Springer，2011：44-51.
[29] CHUNG J，GULCEHRE C，CHO K H，et al.Empirical evaluation of gated recurrent neural networks on sequence modeling[J].arXiv：1412.3555，2014.
[30] LI D，HU J，WANG C，et al.Involution：inverting the inherence of convolution for visual recognition[J].Computer Vision and Pattern Recognition，2021，26（1）：1-12.
[31] LIN T Y，GOYAL P，GIRSHICK R，et al.Focal loss for dense object detection[C]//Proceedings of the 2017 IEEE International Conference on Computer Vision，Venice，2017：2999-3007.
[32] ROSSLER A，COZZOLINO D，VERDOLIVA L，et al.FaceForensics++：learning to detect manipulated facial images[C]//Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision，2019：1-11.
[33] LI Y，YANG X，SUN P，et al.Celeb-DF：a large-scale chal-lenging dataset for deepfake forensics[C]//Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition，2020：3207-3216.