Multimodal Deepfake Detection Using Video Analysis and Social Metadata Signals

The spread of hyper-realistic deepfakes via social networks requires powerful detection systems that can stand the aggressive video compression, which, in most cases, makes the conventional forensic models useless. A new hybrid visual forensics and social signal modeling Dual-Pipeline System is suggested to deal with the sudden performance decreases in low-bitrate in-the-wild content to combine the two. The visual architecture is based on the EfficientNet to extract spatial features with Vision Transformers to model global spatio-temporal consistency, which reduces the effect of compression artifacts. At the same time, to reveal that commenter skepticism is an extrinsic forensic feature with the help of BERT-based sentiment analyzers, a parallel textual pipeline is used, and user comment skepticism is interpreted as substitute evidence. The modalities are combined using a dynamic weighting formula which shifts the dependence of social cues dynamically when there is loss of visual quality. This multimodal fusion is proved to be very useful in boosting detection accuracy with compressed media over unimodal baselines, providing a robust platform to content moderation in the real world.