Extending a Multi-Model Face-Detection and Feature-Extraction Approach to Video: Temporal-Consistency-Aware Deepfake Detection: A Simulation-Based Evaluation

Abstract

Background. A deepfake video is not a single still image; it is a sequence of frames whose per-frame manipulation strength a generative method does not necessarily hold constant, and a detector that only ever looks at one frame at a time, or naively averages across frames, can under-use exactly this frame-to-frame inconsistency as a detection signal. Rahman et al. investigated face-detection and feature-extraction techniques for deepfake detection using a multi-model machine-learning approach; this article extends that approach, and this study series' companion still-image evaluation of it, from single face crops to video, asking what a frame-level classifier built the same way should do with the resulting sequence of per-frame predictions. Methods. Using the strongest detector-and-feature combination identified in the companion still-image evaluation (an MTCNN-style landmark-aligned detector with Xception-style deep-embedding features) and a Random Forest frame-level classifier, this study implements and compares three video-level aggregation strategies: a per-frame majority vote, a mean-probability pool, and an explicit temporal-consistency feature that combines the mean per-frame probability with the frame-to-frame standard deviation of that probability, with every strategy's decision threshold calibrated on training-set videos only. The evaluation uses 1,000 synthetic twelve-frame videos per seed, over five seeds, with fake videos engineered to carry a deliberately weak mean per-frame artifact signal that fluctuates frame to frame, so that no single frame is reliably informative on its own. Results. The temporal-consistency feature reached the highest accuracy (0.863), F1 (0.864), and AUC (0.939) of the three strategies, ahead of mean-probability pooling (0.859 / 0.856 / 0.932) and well ahead of majority voting (0.781 / 0.782 / 0.865); it also achieved this with a materially higher recall than mean-probability pooling (0.872 versus 0.835) at a small precision cost (0.856 versus 0.879). Conclusion. When a deepfake's artifact signal is weak on any single frame but inconsistent across frames, a design this evaluation argues is a realistic description of how imperfect generative synthesis actually behaves, explicitly modelling that frame-to-frame inconsistency as a detection feature outperforms both naive per-frame aggregation strategies, extending Rahman et al.'s multi-model approach usefully from still images to video.