Structure matters: analyzing videos via graph neural networks for social media platform attribution

Abstract

Detecting the origin of a digital video within a social network is a critical task that aids law enforcement and intelligence agencies in identifying the creators of misleading visual content. In this research, we introduce an innovative method for identifying the original social network of a video, even when the video has been altered through actions like group of frames removal and file container reconstruction. The proposed method takes advantage of the video encoding’s temporal uniformity, leveraging motion vectors to characterize the specific features associated to various social media platforms. Each video is represented by a graph where nodes correspond to macroblocks. These macroblocks are interconnected by following the inter-prediction rules outlined in the H.264/AVC codec standard. Such a structure can be then classified using a graph neural network to predict the platform on which the video has been shared. Experimental results demonstrate that this approach outperforms both codec- and content-based approaches, underscoring the effectiveness of a structural approach in attributing the social media platform from which videos originated.

BibTeX

@inproceedings{gemelli2024structure,
    title={Structure Matters: Analyzing Videos Via Graph Neural Networks for Social Media Platform Attribution},
    author={Gemelli, Andrea and Shullani, Dasara and Baracchi, Daniele and Marinai, Simone and Piva, Alessandro},
    booktitle={ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)},
    pages={4735--4739},
    year={2024},
    organization={IEEE},
    doi = {10.1109/ICASSP48485.2024.10447089},
}

Acknowledgments

This work was supported in part by the Italian Ministry of Universities and Research (MUR) under Grant 2017Z595XS, and in part by the Defense Advanced Research Projects Agency (DARPA) under Agreement No. HR00112090136.