Image Forensics through Illumination Cues and Deep Learning
No Thumbnail Available
Date
2022
Authors
Journal Title
Journal ISSN
Volume Title
Publisher
Abstract
This thesis proposes a number of forensics methods for detecting and localizing various types of forgeries present in images. The first method focuses on exposing splicing forgeries involving human faces in the front pose. It utilizes the inconsistencies in the lighting environments (LEs) in different faces present in the image under investigation. A novel LE estimation method is proposed based on a low-dimensional lighting model, created from a set of front pose face images captured under different directional light sources. The limitation of the method is that it can detect splicing forgeries only in images containing front-pose faces. This limitation is overcome by the proposed second method that can detect spliced faces of any pose. It is based on extracting an illumination-signature that captures the information regarding the illumination source colours from all the faces and checking the inconsistencies in them for exposing the spliced faces. The dichromatic plane histogram, which is created by utilizing the dichromatic reflection model and 3D Hough transform, is proposed as the illumination-signature. Although this method can detect spliced faces of any pose, it assumes that the skin colours of different faces are the same and hence fails in images containing persons of different races. To detect spliced faces of any pose and skin colour, a deep learning-based method is proposed, that checks the inconsistencies in the visual features present in the face regions of the illumination map (face-IM) computed from the input image. A siamese convolutional neural network (CNN) is employed for comparing the face-IMs in a pair-wise manner and trained on a set of authentic and artificially-created spliced face-IM pairs. After the training, the CNN part of the siamese network is used for extracting features from the face-IMs. Then, a support vector machine (SVM) is employed for classifying the concatenated features of face-IMs of a pair. In an attempt to detect different types of image editing operations and also to detect and localize various forgeries in a single framework, a universal forensics method is proposed. The method trains a siamese CNN for checking whether a pair of image patches is modified using the same type of editing operation or not. The trained siamese network is then used for classifying images modified using image editing operations present in the training phase and also those not present in the training stage in a one-shot learning framework. Using the trained siamese network, a method is proposed for localizing and detecting various types of forgeries. To further improve the forgery localization performance, a method is proposed, that trains a two-steam encoder-decoder network specifically for localizing different types of forgeries. One stream of the network learns low-level image manipulation-related features and the other stream learns high-level manipulation-related features. The outputs of the decoders of both the streams are fused using the late-fusion technique for generating a single output prediction map.
Description
Supervisor: P. K. Bora
Keywords
ELECTRONICS AND ELECTRICAL ENGINEERING