PRNU-Based Content Forgery Localization Augmented With Image Segmentation

The advances in image editing and retouching technology have enabled an unskilled person to easily produce visually indistinguishable forged images. To detect and localize such forgeries, many image forensic tools rely on visually imperceptible clues, e.g. the subtle traces or artifacts introduced during image acquisition and processing, and slide a regular, typically square, detection window across the image to search for discrepancies of specific clues. Such a sliding-window paradigm confines the explorable neighborhood to a regular grid and inevitably limits its capability in localizing forgeries in a broad range of shapes. While image segmentation that generates segments adhering to object boundaries might be a promising alternative to the sliding window-based approach, it is generally believed that the potential of the segmentation-based detection scheme is hindered by object removal forgeries where meaningful segments are often unavailable. In this work, we take forgery localization based on photo-response non-uniformity (PRNU) noise as an example and propose a segmentation-based forgery localization scheme that exploits the local homogeneity of visually indiscernible clues to mitigate the limitations of existing segmentation approaches that are merely based on visually perceptible content. We further propose a multi-orientation localization scheme that integrates the forgery probabilities obtained with image segmentation and multi-orientated detection windows. The multi-orientation scheme aggregates the complementary strengths of image segmentation and multi-oriented detection window in localizing the object insert and object removal forgeries. Experimental results on a public realistic tampering image dataset demonstrate that the proposed segmentation-based and multi-orientation forgery localization schemes outperform existing state-of-the-art PRNU-based forgery localizers in terms of both region and boundary F1 scores.