© Springer Nature Switzerland AG 2019
Ali Dehghantanha and Kim-Kwang Raymond Choo (eds.)Handbook of Big Data and IoT Securityhttps://doi.org/10.1007/978-3-030-10543-3_9

Internet of Things Camera Identification Algorithm Based on Sensor Pattern Noise Using Color Filter Array and Wavelet Transform

Kimia Bolouri1  , Amin Azmoodeh1, 2  , Ali Dehghantanha1   and Mohammad Firouzmand3  
(1)
Cyber Science Lab, School of Computer Science, University of Guelph, Guelph, ON, Canada
(2)
Department of Computer Science & Engineering, Shiraz University, Shiraz, Iran
(3)
Iranian Research Organization for Science and Technology (IROST), Tehran, Iran
 
 
Kimia Bolouri
 
Amin Azmoodeh
 
Ali Dehghantanha (Corresponding author)
 
Mohammad Firouzmand

Abstract

The Internet of Things (IoT) is cutting-edge technology of recent decade and has influenced all aspects of our modern life. Its significance and wide-range applications necessitate imposing security and forensics techniques on IoT to obtain more reliability. Digital cameras are the noteworthy part of IoT that play a vital role in the variety of usages and this entails proposing forensic solutions to protect IoT and mitigate misapplication.

Identifying source camera of an image is an imperative subject in digital forensics. Noise characteristics of image, extraction of Sensor Pattern Noise (SPN) and its correlation with Photo Response Non-Uniformity (PRNU) has been employed in the majority of previously proposed methods. In this paper, a feature extraction method based on PRNU is proposed which provides features for classification with Support Vector Machine (SVM). The proposed method endeavours to separate more powerful signals which is linked to camera sensor pattern noise by identifying color filter array pattern. To overcome the computational complexity, the proposed method is boosted by utilizing wavelet transform plus reducing dimensions of the image by selecting the most important components of noise. Our experiments demonstrate that the proposed method outperforms in terms of accuracy and runtime.

Keywords

Internet of ThingsForensicCamera identificationPhoto response non-uniformity (PRNU)Sensor pattern noise (SPN)Color filter array (CFA)Wavelet transformSupport vector machine (SVM)

1 Introduction and Background

The Internet of Things (IoT) typically consists a wide-range of Internet-connected smart nodes that sense, store, transfer and process heterogenous collected data [1, 2]. IoT devices are influencing various aspects of life ranging from health [3] and agriculture [4] to energy management [5] and military domains [6]. Its pervasiveness, usefulness and significance have made IoT an appealing target for cyber-criminals [7].

Millions of smart nodes are communicating sensitive and private data through the IoT networks that necessitate protecting IoT against malicious attacks [8]. Above and beyond traditional malware, crypto-ransomware attacks are becoming a challenge for securing IoT infrastructure [9, 10]. IoT-based Botnets (Botnet of Things) and DDoS attacks using an IoT infrastructure are becoming major cyber threat [11, 12]. Therefore, it is important to propose efficient and accurate methods for forensics investigation of IoT networks [13, 14]. Digital cameras are inseparable and substantial subset of IoT nodes that play key role in a wide-range of applications and should possess forensic mechanism to boost its safety and admissibility [15].

Nowadays digital imaging instruments are expanding rapidly and replacing their analog counterparts. Digital images are penetrating all aspects of our life ranging from photography and pharmaceutical pictures to court evidences and the military services [16, 17]. In many applications, validity of images and identification of origin camera are crucial to judge about the images as an evidence. Particularly, electronic image identification techniques are vital in many judiciary processes. For instance, recognizing the source tools could disclose the mystery of felonies. Furthermore, recognizing the source of image is beneficial to every digital forensic approach related to images and videos [18].

To identify source camera of a taken image, it is necessary to have knowledge about what processing actions are performed on the real scene and how it has influenced final digital outcome. Majority of identification techniques have been proposed in this regard. A sizeable proportion of these methods leverage sensor pattern noise because it remains in image as a fingerprint of digital imaging sensor which is highly connected with the type of camera [1922]. Lukas et al. [19] proposed a method which identified type of camera using noise correlation relating to Photo Response Non Uniformity (PRNU). In their method, it was important to collect a large-scale dataset of images relating to specified images for averaging them among the residual noise and obtaining Sensor Pattern Noise (SPN). Moreover, they analysed error rates change using JPEG compression and gamma correction.

Li [23] opined that stronger signal component in an SPN is the less trustworthy component and attenuated it. Then, by assigning weighting factors inversely to the magnitude of the SPN components identified the source of images. Kang et al. [24] examined and compared source camera identification performance and introduced camera reference phase SPN and removed image’s contamination and therefore reached more precise classification. They also presented by theoretical and practical experiments that their proposed method could achieve higher ROC outcome compared with [19].

Quiring and Kirchner [25] studied sensor noise in an adversarial environment and focused on sensor fingerprint that is fragile to lossy JPEG compression and proposed a method to accurately detect manipulated images by attackers. Applying camera identification on user authorization process, Valsesia et al. [26] leveraged high-frequency components of the photo-response nonuniformity and extracted it from raw images. They also presented an innovative scheme for efficient transmission and server-side verification. Sameer et al. [27] introduced a two-level classification mechanism using Convolutional Neural Networks to detect authentic and counter-forensic images and then type of undergone attack.

The reminder of this paper is as follows. Section 2 reviews the structure of digital camera and Sect. 3 explains the proposed method and Sect. 4 evaluates the performance of it. Section 5 completes this paper by discussing and concluding about the achievements of this paper.

2 Digital Camera

In this section, we briefly explain processing stages which occur in a particular digital camera and investigate to find features that facilitate recognizing source camera of images. Imaging stages in digital camera are illustrated in Fig. 1.
../images/451567_1_En_9_Chapter/451567_1_En_9_Fig1_HTML.png
Fig. 1

Stages of imaging in digital camera [22]

Light enters a set of lenses from the scene and is passed through anti-visual filters. Then, it is accessible to color filter array and in each pixel, one of the components of Red (R), Green (G) and Blue (B) is taken into account because this sensor is only able to pass one color in each pixel. Over here, an interpolation process is applied to estimate intensity of two other colors for each pixel using neighborhoods. In the next stage, the sequence of image processing operations like color correction, white color balance, gamma correction, image improvement, JPEG compaction etc., have been performed. Afterwards, the image is saved as a particular image file. Some specifications in this process can be applied for camera type identification such as sensor pattern noise (SPN), camera dependency response function, resampling, color filter array interpolation, JPEG compression and lens aberrations. In this study, we utilize a combination of camera pattern noise and color filter array for recognizing source camera of an image.

2.1 Camera Noise

The noise which is available in images originates from different sources. In order to source camera identification, we explore all types of noise which are caused by camera and has the minor dependency to surrounding environment or the subjects which are imaged.

Based on source of noise, they are categorized as temporal, spatial and a combination of both. The noise which appears in the reconstructed image and is fixed in a definite location of image, indicates Fixed Pattern Noise (FPN). Basically, since it has been fixed from spatial point of view, it can be removed in dark points using signal processing methods. The principle component of FPN is current non-uniformity of the dark points in a charge-coupled device (CCD) sensor. This can be due to photo contact time or high temperature. Complementary metal-oxide semiconductor (CMOS) is the main sources of FPN current non-uniformity of dark points [28]. The considerable point in FPN is that spatial pattern of these changes remains fixed. Since FPN is added to all frames or images produced with a sensor and is independent of light, it can be deleted from the image by reducing a dark frame. The source which is similar to FPN in terms of characteristics is PRNU. In fact, this is the same difference in response to light in pixel when light enters the sensor. One of the reasons is non-uniformity of size of the active areas which throughout light photons are absorbed. PRNU is the result of silicon material heterogeneity and some other effects which have been created during production process.

2.2 Color Filter Array and Bayer Pattern

As described in camera imaging stages, light enters Color Filter Array (CFA) sensors after passing lens and anti-visual filters. This array is two-dimensional and monochromic. In other words, sensors pass only one color in each pixel (Fig. 1). In order to obtain three color channels, an interpolation function should be applied and two subsequent colors for each pixel should be identified. Color pattern is different due to various types of CFA in different cameras. Among these patterns, Bayer Pattern [29] is leveraged frequently and includes four dimensional matrix which is repeated during CFA. This pattern includes two green pixels, one blue pixel and one red pixel and has four different models dependent on pixel location (Fig. 2). Considering that human vision is more sensitive to green color, most pixels are allocated to green color in this pattern.
../images/451567_1_En_9_Chapter/451567_1_En_9_Fig2_HTML.png
Fig. 2

Bayer pattern

It is obvious that the signal which is located in CFA is more powerful and since it is directly received from the sensor is an appropriate option to be used for extraction of features and type detection of digital camera. Therefore, this pattern can be applied as a fingerprint to identify source of image. In fact, it can be recognized that the sampled image in each pixel has been entered directly from sensor of camera or has been interpolated.

3 Proposed Method

Considering the fact that different cameras possess various noise characteristics due to their sensor specifications, disparate patterns should be obtained from the images. To recognize images’ source, the proposed method includes two main building block namely Feature Extraction and PRNU Pattern Classification.

3.1 Feature Extraction

PRNU noise is extracted by de-noising filter that is rooted in wavelet transform. De-noising filter is explained in [30]. This filter of image extracts Gaussian noise with a given variance (which enters the filter as an input). Based on this assumption, image and noise are additive composition of non-static Gaussian signal which have a given variance in wavelet range. We obtain residual noise of image by subtracting de-noised image from original image (Eq. (1)).

$$\displaystyle \begin{aligned} \begin{aligned} {PRNU=I-I_{denoised}}\\ \end{aligned} \end{aligned} $$
(1)
I denoised is obtained with wavelet de-noising filter[30] and consequently, we are able to compute high frequency components of image which have been lost in the de-noised image as image noise pattern. It is to be noted that complexities and edges of image are also effective on the obtained pattern. In this method, PRNU and image dimensions are same. In the subsequent section three different feature extraction methods are described. Section 3.1.1 explains basic PRNU feature extraction method and Sects. 3.1.2 and 3.1.3 belong to the proposed method.

3.1.1 Basic PRNU

This feature extraction method uses pattern noise of the entire image similar to Lukas et al. [19] algorithm. The difference is that image vector extraction method and vectors classification with SVM classifier have been used instead of using averaging method and calculate camera sensor noise pattern and correlation rate. Moreover, this algorithm employ PCA for dimension reduction. Two main stages to extract desired feature are as follows:
  1. 1.

    Extracting pattern noise across the entire image using Eq. (1).

     
  2. 2.

    Using PCA for dimensionally reduction and eliminating useless components of noise

     
Considering what we explained in Sect. 2.2, all pixels of the image have not been received directly from sensor of camera and some of them are the result of interpolation based on color filter array pattern. Therefore, noise pattern obtained with this method (Stage 1) includes color interpolation errors. In fact, most digital camera identification algorithms which act based on pattern noise, use one of the color channels.

3.1.2 PRNU Extraction Considering Color Channels

In order to reduce color interpolation error and promote pattern noise, in [22] colors analysis in pattern noise and extraction of Color-Decoupled Photo Response Non-Uniformity are used. For this purpose, it has analyzed image into four sub-images for each color channel and extracted PRNU pattern noise using Eq. (1). To compare the proposed algorithm with this algorithm, we use CD-PRNU instead of extracting PRNU in the first stage of feature extraction. The proposed algorithm considers all color channels in extraction of pattern noise as well as CD-PRNU extraction algorithm. It is notable that CFA pattern has not been certainly recognized in CD-PRNU extraction algorithm and the extracted pattern noise is a combination of pattern noise of all three RGB color channels. Instead, CFA pattern is identified in the proposed algorithm using Choi et al. method [31]. In this regard, we only select the pixels from pattern noise which are related to CFA.

A method for identifying color filer array (CFA) which uses Intermediate Value Counting is presented in [31] and has been formed based on the primary hypothesis that CFA interpolation fills empty pixels using neighbor pixels. As described in Sect. 2.2, there are different pixel patterns for coloring green, red and blue channels. For each channel, a special neighborhood is defined and intermediate value is counted. CFA pattern is estimated by information of these intermediate values in three channels. Finally, one of the four Bayer patterns (RGGB, GRBG, BGGR and GBRG) is attributed to any camera.

In the proposed algorithm, we first determine CFA pattern using intermediate value counting method and then in PRNU pattern, select pixels which are related to CFA. It is assumed that PRNU matrix is a noise pattern extracted from image (Eq. (1)). For an image of M*N with three RGB color channels, PRNU matrix will be M*N with three color channels. To select pixels among pattern noise, we present Eqs. (2)–(5).

$$\displaystyle \begin{aligned} \begin{aligned} {CFA_{RGGB}(2x-1 , 2y-1)=PRNU_{R}(2x-1 , 2y-1)}\\ {CFA_{RGGB}(2x-1 , 2y)=PRNU_{G}(2x-1 , 2y)}\\ {CFA_{RGGB}(2x , 2y-1)=PRNU_{G}(2x , 2y-1)}\\ {CFA_{RGGB}(2x , 2y)=PRNU_{B}(2x , 2y)}\\ \end{aligned} \end{aligned} $$
(2)

$$\displaystyle \begin{aligned} \begin{aligned} {CFA_{GRBG}(2x-1 , 2y)=PRNU_{R}(2x-1 , 2y)}\\ {CFA_{GRBG}(2x-1 , 2y-1)=PRNU_{G}(2x-1 , 2y-1)}\\ {CFA_{GRBG}(2x , 2y)=PRNU_{G}(2x , 2y)}\\ {CFA_{GRBG}(2x , 2y-1)=PRNU_{B}(2x , 2y-1)}\\ \end{aligned} \end{aligned} $$
(3)

$$\displaystyle \begin{aligned} \begin{aligned} {CFA_{BGGR}(2x , 2y)=PRNU_{R}(2x , 2y)}\\ {CFA_{BGGR}(2x-1 , 2y)=PRNU_{G}(2x-1 , 2y)}\\ {CFA_{BGGR}(2x , 2y-1)=PRNU_{G}(2x , 2y-1)}\\ {CFA_{BGGR}(2x , 2y)=PRNU_{B}(2x-1 , 2y-1)}\\ \end{aligned} \end{aligned} $$
(4)

$$\displaystyle \begin{aligned} \begin{aligned} {CFA_{GBRG}(2x-1 , 2y-1)=PRNU_{R}(2x-1 , 2y-1)}\\ {CFA_{GBRG}(2x-1 , 2y)=PRNU_{G}(2x-1 , 2y)}\\ {CFA_{GBRG}(2x , 2y-1)=PRNU_{G}(2x , 2y-1)}\\ {CFA_{GBRG}(2x , 2y)=PRNU_{B}(2x , 2y)}\\ \end{aligned} \end{aligned} $$
(5)
Where x𝜖[1, 2, .., M] and y𝜖[1, 2, ..., N] and PRNU R, PRNU G, PRNU B are PRNU matrix in green, red and blue channels respectively. CFA matrix is a two-dimensional M*N matrix which includes values of the pixels directly received from color filter array sensors. In fact, CFA matrix has been obtained from combination of PRNU color channels. Image of this combination for RGGB pattern is shown in Fig. 3. Finally, PCA is applied on calculated CFA matrix to reduce dimension.
../images/451567_1_En_9_Chapter/451567_1_En_9_Fig3_HTML.png
Fig. 3

Obtaining CFA matrix from PRNU

3.1.3 Reduction of Image Dimensions Using Wavelet Transform

Although PCA is applied at the final stage of methods in Sects. 3.1.2 and 3.1.1, high resolution of digital images and consequently processing time remains another problem to solve. In order to mitigate this issue, we leverage Multi-Resolution Theory and Discrete Wavelet Transform [32]. Suppose M*N image as input of algorithm, we will have four sub bands (LL: image estimation, LH: horizontal, HL: vertical and HH: diametrical) as output. Effect of wavelet transform on a test image and bands resulting from it are shown in Fig. 4. Since our goal is to retain pattern noise and reduce dimensions of the image, we use HH which includes the most important parts of pattern noise and perform all stages of the proposed algorithm on HH band of wavelet transform. In this regard, the dimensions of image is 4l-times smaller (l is the level of wavelet transform). It is evident that higher wavelet transform level (the higher value of l), leads to lower dimensions of image and higher speed of image processing. However, exceeding in l decrease the detection identification performance. Figure 5 illustrates how to analyze image in HH band.
../images/451567_1_En_9_Chapter/451567_1_En_9_Fig4_HTML.png
Fig. 4

Wavelet transforms bands in a hypothetical image

../images/451567_1_En_9_Chapter/451567_1_En_9_Fig5_HTML.png
Fig. 5

Wavelet transform bands to level 2

3.2 Noise Classification

Support Vector Machines (SVM) [33] are a set of classification methods relating to supervised learning which analyze data and identify the patterns. To utilize SVM, it is necessary that each of the samples be represented as a vector of real numbers. In our experiments, input part is a vector of features which is extracted from PRNU noise, CFA pattern and its wavelet transform. Furthermore, PCA is applied to reducing dimensions of data and selecting more beneficial elements of data [34]. Output of SVM classifier is a label predicting camera type.

4 Experiments

4.1 Dataset and Settings

Dataset used in this research include images of 6 digital cameras shown in Table 1. each dataset includes 130 images. Photographing conditions are equal in all images. All pictures were taken on a tripod without flash, automatic focus, without magnitude and the best compression quality and also other options have been automatically set on the default values. In addition, attempt has been made to have images with different subjects and non-uniform edges and complexities so as to simulate real-world condition. To assess the validity of our performance evaluation, 10-fold cross-validation is applied. System specification to run experiments is Microsoft Windows7, CPU Intel Corei3 3.20GHz and 4Gb of RAM.
Table 1

Images dataset information

Label

Camera model

Sensor

CFA pattern

Image size

C1

Canon EOs400D

CMOS

RGGB

3888*2592

C2

Kodak EasyShare CX7530

CCD

GBRG

2560*1920

C3

HP PhotoSmart E327

CCD

RGGB

2560*1920

C4

Panasonic DMC-FZ20

CCD

RGGB

3648*2738

C5

Canon PowerShot A400

CCD

GBRG

1536*2048

C6

Panasonic Lumix DMC-FS5

CCD

GRBG

3648*2736

4.2 Results

At the first stage, features explained in Sect. 3.1 are extracted for PRNU [19], CD-PRNU [22] and the proposed algorithm. In the next stage, a SVM classifier is trained for each algorithm based on 10-fold cross-validation algorithm. Accuracy is the performance evaluation metric which is the number of samples that a classifier correctly detects, divided by the number of all samples (Eq. (6)).

$$\displaystyle \begin{aligned} Accuracy=\frac{TP+TN}{TP+TN+FP+FN} \end{aligned} $$
(6)
Results of these three algorithms are shown in Table 2. As can be seen from Table 2, CD-PRNU feature extraction were negligibly accurate compared with PRNU while accuracy has been considerably improved in the proposed algorithm. In order to evaluate effect of applying wavelet transform on runtime, we apply three levels of it (l = 1, 2, 3) on the images to reduce processing time and then use HH part of wavelet transform instead of the image. Implementation results are shown in Table 3 and Fig. 6 compares runtime of different methods. It is obvious although higher wavelet transform level leads to lower accuracy in identification, the runtime speed increases significantly due to reduction of the image size in higher levels of wavelet transform.
../images/451567_1_En_9_Chapter/451567_1_En_9_Fig6_HTML.png
Fig. 6

Algorithm comparison in terms of time

Table 2

Accuracy obtained from the proposed algorithm and comparison with previous methods

 

PRNU [19]

CD-PRNU [22]

Proposed method

C1

91%

89.00%

99.50%

C2

83.00%

83.33%

91.83%

C3

84%

90.00%

97.17%

C4

91.83%

94.67%

100%

C5

90%

86.83%

99.50%

C6

94.67%

91.67%

99.83%

Average

89%

89.25%

97.97%

Table 3

Results of wavelet transform application and image dimensions reduction

 

l = 1

l = 2

l = 3

C1

99.33%

100%

100%

C2

85.83%

84.17%

83.50%

C3

86.67%

83.67%

83.17%

C4

97%

89%

82.83%

C5

98.17%

84.50%

84.83%

C6

99.17%

100%

100%

5 Conclusion

Nowadays, the Internet of Things penetrating all aspect of our life and enormous digital images are taken in small intervals by IoT nodes for different applications. This soar in usage and importance of IoT necessitates proposing applicable method for forensic issues. Camera identification refers to a category of methods that endeavour to recognize source camera based on images taken by it.

In this paper, an algorithm has been proposed for extracting features so as to digital camera identification. The proposed algorithm has been evaluated by a dataset including images taken by six different digital cameras. To increase accuracy, we proposed a new feature extraction method to generate CFA pattern. Furthermore, to select the principle components of pattern noise and to reduce dimensions of pattern noise, we leveraged HH part of wavelet transform and results demonstrate upsurge of runtime speed. The proposed method outperform 97.07% of accuracy which are higher than the average result obtained in the rival methods, 89% and 89.25% from PRNU [19] and CD-PRNU [22] respectively. To outdo other methods in terms of runtime, we contributed that speed considerably will increases using HH component of wavelet transform while average accuracy remains approximately above 90%.