Towards open-world multimedia forensics through media signature encoding

Abstract

In this paper we introduced a framework for the forensic analysis of multimedia in open-world settings. We exploited a siamese architecture based on denoising autoencoders to encode multiple forensic features from different domains (content- and container-based features) into a compact descriptor. The proposed method is designed to cluster media belonging to similar toolchains in the signature space. We demonstrated the effectiveness of the proposed method by analysing two meaningful experimental setups involving both digital images and videos. Experimental results highlighted that the method is capable of clustering correctly media belonging to unfamiliar processing toolchains, thus allowing the identification of new and previously unknown life cycles. We also found that, when the unknown toolchain partially share the life cycle with one or more available toolchains, a non-marginal degree of compatibility is maintained in the encoded space, thus providing clues on the relevant life cycle. Finally, the proposed method has the potential to scale to internet volumes of information, given its capability to encode features in a low-dimensional space with limited computational effort.

This work can be considered a first step towards the design of a bigger picture for the investigation of media in open-world settings. Similarly to former fusion frameworks, the suggested method’s primary drawback is that it is mostly dependent on the features that are fed into the network. In fact, different features might be more or less relevant in capturing traces left by new possible tampering operations, and their initial choice may have an impact on the overall capability of the system. Future research ought to focus on assessing the framework’s robustness in terms of feature selection in characterizing unseen manipulations. Additionally, exploring the potential of incorporating new media types like digital audio and studying novel mathematical tools for signature generation can further enhance the framework’s effectiveness.

BibTeX

@article{baracchi2024towards,
    title={Towards open-world multimedia forensics through media signature encoding},
    author={Baracchi, Daniele and Boato, Giulia and De Natale, Francesco and Iuliani, Massimo and Montibeller, Andrea and Pasquini, Cecilia and Piva, Alessandro and Shullani, Dasara},
    journal={IEEE Access},
    year={2024},
    publisher={IEEE},
    doi={10.1109/ACCESS.2024.3391809}
}

Acknowledgments

This work was supported in part by the Italian Ministry of Universities and Research (MUR) under Grant 2017Z595XS, and in part by the Defense Advanced Research Projects Agency (DARPA) under Agreement No. HR00112090136.