Forensic Camera Classification: Verification of Sensor Pattern Noise Approach

Nitin Khanna
Graduate Student
School of Electrical and Computer Engineering
Purdue University
West Lafayette, Indiana

Aravind K. Mikkilineni
Graduate Student
School of Mechanical Engineering
Purdue University
West Lafayette, Indiana

Edward J. Delp
Professor
School of Electrical and Computer Engineering
Purdue University
West Lafayette, Indiana

Abstract | Introduction | Digital Camera Overview | Correlation-Based Approaches | Experimental Results | Conclusions | Acknowledgments | References

Abstract

Digital images can be obtained through a variety of sources, including digital cameras and scanners. In many cases, the ability to determine the source of a digital image is important. This paper verifies the method presented in Lukas et al. (2005a, b) for authenticating images acquired using digital still cameras. The method is based on imaging sensor pattern noise. A reference pattern is estimated for each camera and is treated as a unique fingerprint of that camera. To identify the source camera of an unknown image, the noise extracted from the image is correlated with all of the reference camera patterns obtained from training images. This correlation-based approach is shown to have close to 100 percent classification accuracy.

Introduction

Advances in digital imaging technologies have led to the development of low-cost and high-resolution digital cameras and scanners, both of which are becoming ubiquitous. Digital images generated by various sources are widely used in a number of applications, from medical imaging and law enforcement to banking and daily consumer use. Along with the proliferation of digital content, the functionality of image-editing software is also escalating, allowing even an amateur to easily manipulate digital images. Under these circumstances, digital images can be used as “legal” only if they can be properly authenticated. Forensic tools that help establish the origin, authenticity, and chain of custody of such digital images are essential to a forensic investigator. Therefore, a reliable and objective way to examine digital image authenticity is needed.

The image source identification problem can be addressed at various levels. Investigators may want to find the particular device (digital camera or scanner) that generated the image, or they might be interested in knowing only the make and model of the device. A number of robust methods have been proposed for source-camera identification (Bayram et al. 2005; Farid and Popescu 2005; Geradts et al. 2001; Kharrazi et al. 2004; Lukas et al. 2005a, b; Lukas et al. 2006a, b). Khanna et al. (2007) presented a novel technique for classifying images based on their sources, both scanned and unscanned.

One approach for digital camera identification is based on characterizing the imaging sensor used in the device. Geradts et al. (2001) have shown that defective pixels can be used for reliable camera identification, even from lossy compressed images. This type of noise, generated by hot or dead pixels, is typically more prevalent in inexpensive cameras and can be visualized by averaging multiple images from the same camera. These errors can remain visible even after the image is compressed. However, many cameras post-process the captured image to remove these types of noise, so this technique cannot always be used.

Lukas et al. (2005a, b; 2006a, b) did the pioneering work to develop source-camera identification techniques using the pattern noise of the imaging sensor. The identification is based on pixel nonuniformity noise, which is a unique characteristic of both CCD (charged-coupled device) and CMOS (complementary metal oxide semiconductor) sensors. Reliable identification is possible even from images that are resampled and compressed into JPEG (Joint Photographic Experts Group) images.

Pattern noise is caused by several factors, such as pixel nonuniformity, dust specks on the optics, optical interference, and dark current (Holst 1998; Janesick 2001). The high-frequency part of the pattern noise is estimated by subtracting a denoised version of the image from the original. This is performed using a wavelet-based denoising filter (Mihcak 1999).

The reference pattern of a camera is determined by averaging the noise patterns from multiple images obtained from the camera. The reference pattern serves as an intrinsic signature of the camera. To identify the source camera, the noise pattern from an image is correlated with the known reference patterns from a set of cameras. The camera corresponding to the reference pattern giving maximum correlation is chosen as the source camera.

Lukas et al. (2006a, b) extended this scheme to detect forgery in digital camera images. Chen et al. (2007; 2008) presented an improved method for source-camera identification based on the joint estimation and detection of the camera photo-response non-uniformity in images. In this paper the methods implemented use the simple averaging method presented in Lukas et al. (2005a, b); thus further improvements in results may be obtained by using the improved method for sensor noise estimation presented in Chen et al. (2007; 2008).

In this paper, we verify the results presented in Lukas et al. (2005a, b) using a different set of cameras. We performed all of the experiments without making any contact with the authors of the earlier work.

Digital Camera Overview

Imaging Pipeline

The basic elements of a digital camera imaging pipeline are shown in Figure 1. Even though the exact design details and algorithms differ among manufacturers and models, the basic structure of the digital camera pipeline remains the same.

Figure 1: Imaging pipeline for a digital camera

First, light from a scene enters the camera through a lens and passes through a set of filters including an antialiasing filter. Next, the light is “captured” by a sensor. These sensors, typically CCD or CMOS imaging sensors, are color-blind in the sense that each pixel captures only intensity information from the light hitting it. To capture color information, the light first passes through a color filter array (CFA), which assigns each pixel on the sensor one of three (or four) colors to be sampled. Shown in Figure 2 are CFA patterns using RGB (red, green, blue) and YMCG (yellow, magenta, cyan, green) color spaces, respectively, for a four-by-four block of pixels. The individual color planes are filled in by interpolation using the sampled pixel values using a number of different interpolation algorithms depending upon the manufacturer.

Figure 2: Color filter array patterns using RGB and YMCG color spaces, respectively, for a four-by-four block of pixels

Next, a number of operations, which include white-point correction and gamma correction, are performed by the cameras. The image is finally written into the camera memory in a user-specified image format (e.g., RAW, JPEG, or TIFF [tagged image file format]).

Although these operations and stages are standard in a digital camera pipeline, the exact processing details in each stage vary between manufacturers and even between different camera models from the same manufacturer. This variation from one camera model to another can be used to determine the type of camera used to acquire a specific image.

Sensor Noise

The process used to manufacture imaging sensors introduces various defects that create noise in the pixel values (Holst 1998; Janesick 2001). These noises can be classified into three classes based on their origins and effects. First is shot noise, or random noise, which is present in any exposure. Shot noise is not useful in source-camera identification because it varies between frames.

The second type of noise is pattern noise, which refers to any spatial pattern that does not change significantly from image to image. Major sources of pattern noise are point defects, fixed pattern noise (FPN), and photo-response non-uniformity (PRNU).

Point defects, such as hot or dead pixels, are typically compensated for by the camera during postprocessing. Fixed pattern noise is an additive noise caused by stray currents, known as dark currents, from the sensor substrate into the individual pixels. Dark current is a function of detector size, doping density, and foreign matter trapped during fabrication and is independent of incident light intensity.

Photo-response non-uniformity is the variation in pixel sensitivity caused by variations between pixels such as detector size, spectral response, coating thickness, and other imperfections created during the manufacturing process. These variations can be modeled as per-pixel multiplicative factors. Both FPN and PRNU remain the same across multiple frames but vary across different sensors.

The third type of noise is readout noise, which is introduced when reading data from the sensor. It is most visible at high ISO (International Organization for Standardization) or when the image has been brightened excessively.

Whereas noise is generally thought of as an additive term, as is the case with FPN, PRNU is a multiplicative factor. This view of the noise is well established in imaging-sensor literature, and we use the same terminology in this paper as well. Frame averaging will reduce the noise sources except FPN and PRNU. Although FPN and PRNU are different, they are collectively known as scene noise, pixel noise, pixel nonuniformity, or, simply, pattern noise (Holst 1998).

In this paper we focus on using pattern noise for source-camera identification. However, it is extremely difficult to obtain the pattern noise by direct methods such as flat fielding. This is because in most general-purpose cameras, the raw sensor data are unavailable. Furthermore, a number of nonlinear image-processing algorithms are applied to transform voltage sensed by the imaging sensor to the final JPEG or TIFF image data.

So, in the absence of any direct method, indirect methods of estimating the sensor noise are used. In Lukas et al. (2005a, b), a method of estimating sensor noise is successfully used for source-camera identification. This proposed method uses a wavelet filter (Mihcak 1999) in combination with frame averaging to estimate the pattern noise in an image.

Correlation-Based Approaches

Digital cameras mostly use CCD/CMOS imaging sensors. The pattern noise of the imaging sensor has been used successfully for source-camera identification as described in Lukas et al. (2005a, b). The high-frequency part of the pattern noise is estimated by subtracting a denoised version of an image from the original image.

As in Lukas et al. (2005a, b), we use a wavelet-based denoising filter (Mihcak 1999) for denoising the image. The reference pattern of a camera is determined by averaging the noise patterns from multiple images captured by the camera. This reference pattern serves as an intrinsic signature of the camera (Figure 3). The steps shown in Figure 3 are independently applied to all of the cameras in our training dataset, and a set of camera reference patterns is obtained.

Figure 3: Classifier training for correlation-based approach

Figure 4 shows the steps involved in source-camera identification from an unknown image. The first step is to estimate the sensor noise in the test image by using the same denoising filter used in the training phase. Then, this noise pattern is correlated with the camera reference patterns obtained in the training step. Finally, the camera corresponding to the reference pattern with the highest correlation is chosen as the source camera (Lukas et al. 2005a, b).

Figure 4: Source-camera identification using a correlation-based detection scheme

There have been some recent improvements in the sensor-noise-based methodology proposed in Lukas et al. (2005a, b). These improvements show better estimation of camera reference patterns using smaller training sets and improved classification.

Chen et al. (2007; 2008) presented an improved method for source-camera identification based on joint estimation and detection of the camera PRNU in images. Postprocessing operations applied on the estimated reference patterns reduced the positive correlation between reference patterns from cameras of the same make and model and thus further reduced the false-alarm rates in distinguishing cameras with the same make and model.

The methods implemented in the present study use the simple averaging method (Lukas et al. 2005a, b); thus further improvements in results, particularly error analysis showing low false-alarm rates, may be obtained by using the improved method for sensor-noise estimation presented in Chen et al. (2007; 2008).

Another improvement is obtained by replacing correlation with crosscorrelation and using peak-to-correlation energy (PCE) as part of the decision criteria (Goljan and Fridrich 2008). This eliminates the problems with finding experimental thresholds for correlation values and instead uses Neyman-Pearson hypothesis testing for computing the threshold based on a chosen false-alarm rate. Source-camera identification using sensor noise is extended to detect forgery of digital camera images (Lukas et al. 2006a, b).

Let I^k denote the k^th input image of size M by N pixels (M rows and N columns). Let I^knoise be the noise corresponding to the original input image, I^k, and let I^kdenoised be the result of applying a denoising filter on I. Then, as in Lukas et al. (2005a, b),

Let K be the number of images used to obtain the reference pattern of a particular digital camera. Then the two-dimensional array reference pattern is obtained as

Correlation is used as a measure of similarity between the camera reference pattern and the noise pattern of a given image (Lukas et al. 2005a, b). Correlation between two vectors X and Y Є R^N is defined as

where (A•B) represents the vector dot product between vectors A and B, and ║A║ represents the Euclidean norm of vector A.

This correlation is used for source-camera identification from an unknown image. The camera corresponding to the reference pattern giving the highest correlation is chosen as the source camera. An experimental threshold can also be determined, and then the camera corresponding to the reference pattern with a correlation value higher than the threshold will be chosen as the source camera.

Experimental Results

Table 1 lists the digital still cameras used in our experiments. Each of these cameras is used to capture pictures at different resolutions and image-quality settings, with all other settings—such as auto focus, red-eye correction, and white balance—left to default. Pictures taken with these cameras have similar as well as dissimilar contents.

Table 1: List of digital cameras used in experiments

Figure 5 shows a sample of the images used in this study. The image dataset includes a broad range of images, from natural scenes to buildings, and images with different backgrounds, light intensities, and so on.

Figure 5: A sample of the images used in this study

Reference Camera Pattern Generation

Reference camera patterns are obtained by averaging the noise extracted from multiple images from the same camera. It is not necessary to possess the cameras because only the training images are needed and no internal design parameters need to be accessed. To determine the optimal number of training images, reference patterns were generated using varying numbers of training images. These reference patterns were then correlated with a fixed set of 20 randomly chosen test images. The average correlations (ρ_avg) between the camera reference pattern generated from N_ρ training images and the 20 test images are plotted in Figure 6. The number of training images used to estimate the reference patterns is shown along the x-axis and the corresponding average correlation is shown along the y-axis.

Figure 6: Average correlation, ρ_avg, as a function of the number of images, N_ρ, used for calculating the reference pattern

Because the correlation detector is highly sensitive to geometrical transformations such as rotation and it may not be known which way the user held the camera, we needed to incorporate these causes of desynchronization when obtaining the correlation. After estimating the noise, we rotated it both positive and negative 90 degrees and then used the higher of the two correlation values.

Image Identification from Unprocessed Images

For the following experiments on unprocessed images, we estimated camera reference patterns using 200 randomly chosen training images. Figures 7–11 show the correlation values for various images from a camera compared with the reference patterns from all cameras.

Figure 7: Correlation of noise from c1 with 11 reference patterns

Figure 8: Correlation of noise from c2 with 11 reference patterns

Figure 9: Correlation of noise from c5 with 11 reference patterns

Figure 10: Correlation of noise from c9 with 11 reference patterns

Figure 11: Correlation of noise from c10 with 11 reference patterns

We used 11 reference patterns corresponding to 10 different source cameras. For camera c3, we used two reference patterns, one obtained from images captured at a resolution of 3072 by 2304 and the second one obtained from images captured at a resolution of 2048 by 1536. This is to see the effect of sizes of the reference patterns on source-camera identification. To estimate the correlation between noise patterns of different sizes, the larger of the two is always cropped from the top-left corner to match the size of the smaller one.

As described in the section Effect of Resampling on Image Identification (below), we resized the images before denoising to match the size of the reference patterns. In all experiments, we chose the source camera based on the reference pattern giving the highest correlation value. In all cases, the classification accuracy is greater than 98 percent. The first 200 images correspond to those used for estimating the reference pattern, and the rest are used for testing.

It should be noted that even though the correlation between noise patterns from test images and the correct reference pattern is comparatively less than the correlation between noise patterns from training images and the correct reference pattern, the correlation value with the correct reference pattern is much higher than that with the incorrect reference patterns. Correlation with the correct reference pattern is much lower for images of the night sky or those obtained by closing the lid of the camera lens. This observation is consistent with all of the cameras.

Effect of JPEG Compression on Image Identification

In this set of experiments, we analyzed the effect of JPEG compression on source-camera identification. Because the noise extracted using the wavelet-based denoising filter corresponds to high spatial frequencies and the JPEG compression typically discards high-frequency information, the correlation between image noise and the reference patterns is expected to decrease. Experiments on different cameras show that this is indeed true. At the same time, the correlation with the wrong reference patterns also decreased, and accurate source-camera identification was still possible.

Figures 12–15 show the variation in mean and variance of correlation between test images stored at different JPEG quality factors and the reference patterns from correct and incorrect cameras.

Figure 12: Mean and standard deviation of _ρ as a function of the JPEG quality factor for c1 compared with all cameras

Figure 13: Mean and standard deviation of _ρ as a function of the JPEG quality factor for c2 compared with all cameras

Figure 14: Mean and standard deviation of _ρ as a function of the JPEG quality factor for c9 compared with all cameras

Figure 15: Mean and standard deviation of _ρ as a function of the JPEG quality factor for c10 compared to all cameras

Effect of Resampling on Image Identification

This section investigates the possibility of identifying images obtained at a resolution lower than the native resolution of the camera (maximal resolution supported by the camera). We captured 350 images at a resolution of 1024 by 768 using camera c1. For source-camera identification, we assumed that these images had been captured at this lower resolution or rescaled in the computer but not cropped. The camera patterns estimated from images at the native resolution of the camera are used. For cases when the size of the noise pattern of an image does not match the size of a reference camera pattern, we resampled the image using “bicubic” interpolation. Noise is then extracted from the resampled image and correlated with the known camera reference patterns to determine the source camera.

As Figure 16 shows, source-camera identification is possible even from images captured at nonnative resolutions. In Figure 16, the reference pattern c1–1 is estimated by averaging 200 images captured at a resolution of 1024 by 768, whereas the reference pattern c1-2 is estimated by averaging 200 images captured at a resolution of 2048 by 1536. Because most digital cameras do not use simple resampling methods such as bicubic interpolation to obtain lower-resolution images, the correlation of noise extracted from a 1024-by-768-size image is expected to be higher with the reference pattern c1-1 than with reference pattern c1-2. This is indeed true in Figure 16. Further, the correlation with the reference pattern c1-2 is consistently higher than the correlation with any other reference pattern. Thus, even for the cases when sizes of training and testing images are different, the classification accuracy is close to 100 percent, although with a smaller tolerance and, thus, lesser reliability.

Figure 16: Identification of low-resolution (1024 by 768) c1 Canon SD200-1 images

Experiments to see the effect of simultaneous application of JPEG compression and resampling showed similar decline in correlation values but maintained close to 100 percent classification accuracy.

Effect of Malicious Processing

Next we address the issue of preventing source-camera identification by removing the pattern noise from an image. In the experiments performed here, we extracted noise from the denoised image and correlated it with the reference patterns obtained from the initial training images (as described in the earlier sections). Figures 17–19 show the correlation values for various denoised images from a camera compared with the reference patterns from all other cameras. Comparing with Figures 8–11, the correlation values for images that have undergone malicious processing by removing the noise are less than the correlation values when compared with unprocessed images. Even then the classification accuracy remains greater than 98 percent.

Figure 17: Correlation of denoised c2 Canon SD200-2 images with reference patterns from all of the cameras

Figure 18: Correlation of denoised c9 Panasonic DMCFZ4-1 images with reference patterns from all of the cameras

Figure 19: Correlation of denoised c10 Panasonic DMCFZ4-2 images with reference patterns from all of the cameras

Conclusions

In this paper we verified the results presented in Lukas et al. (2005a, b). Figures 7–11 show that use of sensor noise and correlation detection can give close to 100 percent classification accuracy. Further improvements in classification with fewer training images can be achieved by using the improved method for reference-pattern estimation presented in Chen et al. (2007; 2008).

In the present implementation, the threshold used for the correlation detector had to be determined experimentally, which can be avoided by using peak-to-correlation energy (PCE) (Goljan and Fridrich 2008). The effect of postprocessing operations such as JPEG compression, resampling, and noise removal is shown in Figures 12–19. Although there is a decrease in correlation with the correct camera reference pattern, correlations with incorrect reference patterns also decrease with these operations, making successful source-camera identification possible.

These results show the robustness of the correlation-based classification scheme presented in Lukas et al. (2005a, b). Our results also demonstrate potential problems when this scheme is used with postprocessed images as well as the nonoptimal nature of the denoising algorithm used here. Our experiments support the results presented in the original paper. We performed these experiments on our image database using a different set of cameras and without making any contact with the authors of the previous paper while developing our implementation.

Because the experiments performed here were on a different set of cameras, different decision thresholds had to be determined experimentally. The camera-dependent nature of these thresholds had already been indicated in Lukas et al. (2005a, b). The separation of correlation values for some cameras is higher than that for others (for example, compare Figures 7 and 11), and it is possible to discriminate between digital cameras of the same make and model.

Acknowledgments

This material is based upon work supported by the National Science Foundation under Grant No. CNS-0524540. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the National Science Foundation.

References

Bayram, S., Sencar, H., Memon, N., and Avcibas, I. Source camera identification based on CFA interpolation. In: Proceedings of the IEEE International Conference on Image Processing. IEEE, Piscataway, New Jersey, 2005, pp. 69–72.

Chen, M., Fridrich, J., and Goljan, M. Digital imaging sensor identification (further study). In: Proceedings of the SPIE International Conference on Security, Steganography, and Watermarking of Multimedia Contents IX, vol. 6505, no. 1. E. J. Delp III and P. W. Wong, Eds. SPIE Press, Bellingham, Washington, 2007, p. 65050P.

Chen, M., Fridrich, J., Goljan, M., and Lukas, J. Determining image origin and integrity using sensor noise. In: IEEE Transactions on Information Forensics and Security, volume 3, no.1. IEEE, Piscataway, New Jersey, March 2008, pp. 74–90.

Geradts, Z. J., Bijhold, J., Kieft, M., Kurosawa, K., Kuroki, K., and Saitoh, N. Methods for identification of images acquired with digital cameras. In: Enabling Technologies for Law Enforcement and Security, vol. 4232, no. 1. S. K. Bramble, E. M. Carapezza, and L. I. Rudin, Eds. SPIE Press, Bellingham, Washington, 2001, pp. 505–512.

Goljan, M. and Fridrich, J. Camera identification from cropped and scaled images. In: Proceedings of the SPIE International Conference on Security, Steganography, and Watermarking of Multimedia Contents X, vol. 6819, no. 1. E. J. Delp III and P. W. Wong, Eds. SPIE Press, Bellingham, Washington, 2008, p. 68190E.

Holst, G. C. CCD Arrays, Cameras, and Displays. Second Edition. JCD Publishing & SPIE Press, Winter Park, Florida, 1998, pp. 123–144.

Janesick, J. R. Scientific Charge-Coupled Devices. SPIE Press, Bellingham, Washington, January 2001, pp. 605–719.

Khanna, N., Mikkilineni, A. K., Chiu, G. T. C., Allebach, J. P., and Delp, E. J. Forensic classification of imaging sensor types. In: Proceedings of the SPIE International Conference on Security, Steganography, and Watermarking of Multimedia Contents IX, vol. 6505, no. 1. E. J. Delp III and P. W. Wong, Eds. SPIE Press, 2007, p. 65050U.

Kharrazi, M., Sencar, H. T., and Memon, N. D. Blind source camera identification. In: Proceedings of the IEEE International Conference on Image Processing. IEEE, Piscataway, New Jersey, 2004, pp. 709–712.

Lukas, J., Fridrich, J., and Goljan, M. Determining digital image origin using sensor imperfections. In: Proceedings of the SPIE International Conference on Image and Video Communications and Processing, vol. 5685, no. 1. A. Said and J. G. Apostolopoulos, Eds. SPIE Press, Bellingham, Washington, 2005a, pp. 249–260.

Lukas, J., Fridrich, J. J., and Goljan, M. Digital “bullet scratches” for images. In: IEEE International Conference on Image Processing, vol. III. IEEE, Piscataway, New Jersey, September 2005b, pp. 65–8.

Lukas, J., Fridrich, J. J., and Goljan, M. Digital camera identification from sensor pattern noise. In: IEEE Transactions on Information Forensics and Security, vol. 1, no. 2. IEEE, Piscataway, New Jersey, June 2006a, pp. 205–214.

Lukas, J., Fridrich, J., and Goljan, M. Detecting digital image forgeries using sensor pattern noise. In: Proceedings of the SPIE International Conference on Security, Steganography, and Watermarking of Multimedia Contents VIII, vol. 6072, no. 1. E. J. Delp III and P. W. Wong, Eds. SPIE Press, Bellingham, Washington, 2006b, p. 60720Y.

Mihcak, M. K., Kozintsev, I., Ramchandran, K., and Moulin, P. Low-complexity image denoising based on statistical modeling of wavelet coefficients. In: IEEE Signal Processing Letters, vol. 6, no. 12. IEEE, Piscataway, New Jersey, 1999, pp. 300–303.

Popescu, A. and Farid, H. Exposing digital forgeries in color filter array interpolated images. In: IEEE Transactions on Signal Processing, vol. 53, no. 10. IEEE, Piscataway, New Jersey, 2005, pp. 3948–3959.