Home About Us Laboratory Services Forensic Science Communications Back Issues April 2008 Research Research and Technology - Forensic Science...
Info
This is archived material from the Federal Bureau of Investigation (FBI) website. It may contain outdated information and links may no longer function.

Research and Technology - Forensic Science Communications - April 2008

Research and Technology - Forensic Science Communications - April 2008
fsc_logo_top.jpg
fsc_logo_left.jpg

April 2008 - Volume 10 - Number 2

 

Research and Technology

The Magna Database: A Database of Three-Dimensional Facial Images for Research in Human Identification and Recognition

Martin Paul Evison
Director, Forensic Science Program
Associate Professor, Department of Anthropology
University of Toronto at Mississauga
Mississauga, Ontario, Canada

Richard W. Vorder Bruegge
Supervisory Photographic Technologist
Operational Technology Division
Federal Bureau of Investigation
Quantico, Virginia

Introduction | Facial Image Data Collection | Contents and Structure of the Magna Database | Value in Research | Arrangements for Dissemination | References

Introduction

This paper describes the collection and initial application in research of a database of three-dimensional (3-D) facial images. The database is available for research in crime prevention and detection—with particular value in human identification and recognition—and is presently in use in research being undertaken by the authors and researchers in the United States, United Kingdom, and Canada. 

The Need for Forensic Identification of People from Images

The early 21st century has seen an exponential growth in the use of cameras and the ease with which photographic images can be transmitted and disseminated. Not only have security concerns in a post-September 11 world led to more surveillance cameras in public and private places, but technological advances and miniaturization have made it possible for billions of individuals to carry a camera with them at all times. Regardless of whether such cameras are part of closed-circuit television (CCTV) security systems or cell-phone cameras, the increase in their numbers means that more and more criminals are being photographed in the commission of their crimes or during their travels to and from their crimes. As a result, these images are being used more and more often to link suspects to their crimes.

In most criminal cases, the identity of individuals depicted in such photographs is established through the testimony of witnesses who know the subject of the photograph through personal experience—perhaps as a friend, acquaintance, or relative—or as the victim of a crime, such as a bank teller who was robbed or a convenience store clerk who was beaten. From a legal perspective, the combination of the photograph and such “recognition” testimony may be better than having had a person witness the event directly, because the photograph provides physical proof of the event supporting the testimony, whereas an eyewitness has only his or her memory. This type of testimony has been offered in courts almost as long as photography has existed, and it is rarely excluded.

Despite the value of recognition testimony, it often happens that the ability of a witness to identify the subject is challenged, or there may be no witness who can testify to the subject’s identity. In this situation, the identity of the subject depicted in the photograph becomes a scientific question to be addressed by an expert, and there will be a need to establish the scientific foundation for identification of individuals from photographs. Such a foundation supports comparison of photographs not only in criminal cases but also in intelligence and border-security cases in which subjects need to be verified against passport or identification-card photographs. The project described in this paper was implemented as a systematic approach to building a foundation using statistics derived from a large population.

Brief History of Forensic Facial Comparisons

The history of forensic facial comparisons extends back for many decades. The earliest public record of their use in criminal proceedings dates back to November 6, 1970, with a ruling by the U.S. Court of Appeals, Ninth Circuit, in the case of United States v. John Donald Cairns (No. 26095). In that case, the court held that “[t]estimony of photographic identifications specialist in armed bank robbery prosecution, comparing photograph taken by bank’s surveillance camera at time of robbery and police photograph of defendant taken ten days prior to trial, was admissible as aid to jury over objection that testimony invaded province of jury.”  On June 10, 1974, in United States v. Tommy Louis Brown, Virgil David Swain and Robert Lee Nobles (Nos. 73-2279, 73-2678, 73-2280), the Ninth Circuit again upheld the admissibility of facial comparison testimony, as long as it would aid the jury. Finally, the U.S. Second Circuit upheld the admissibility of facial comparisons in United States v. Henry Stuart Brown (No. 468, Docket 74-1947, U.S. Court of Appeals, Second Circuit, February 20, 1975).

There is a small but growing body of literature on forensic facial comparisons. One early technical report on the subject was an article titled “Laboratory Examinations of Photo-Related Evidence” that appeared in the FBI Law Enforcement Bulletin in May 1972 (Federal Bureau of Investigation 1972). Subsequent work has been reported by multiple authors, including Catterick (1992); Evison (2000, 2005); Evison et al. (2006); Mallett (2006); Vanezis and Brierley (1996); Vorder Bruegge (2002); Vorder Bruegge and Musheno (1996); and Yoshino et al. (1996, 2000, 2001).

Finally, it is also significant that the forensic community has recognized facial comparisons as a valid discipline, subject to training, competency and proficiency standards, and best practices (Scientific Working Group on Imaging Technology [SWGIT] 2001, 2005; Scientific Working Group on Digital Evidence [SWGDE]/SWGIT 2004).

Methods of Forensic Facial Comparisons

The most common means of identifying people from photographs involves facial comparison. Photographic comparisons of individuals may be conducted that involve other parts of the body such as the ears and hands, but facial comparisons are, by far, the most common. Methods currently employed in forensic facial comparison include comparative measurement from facial images (“photogrammetric approach”), anthroscopy (qualitative examination of facial features), and image superimposition.

The photogrammetric approach involves limited quantification. Typically, two sets of three or more near-parallel lines are drawn through facial features—e.g., the jawline, the pupils, the nasal bridge—on the offender image. These are compared with a similar set on the suspect image. An assumption is made that the two 2-D images are taken, effectively, from the same pose angle or that pose angle has no significant effect. The two sets are compared for congruence in shape and proportionality in order to establish a match or exclusion.

The probability of a match with another random individual is unknown in the photogrammetric approach, so the significance of a “match” is undefined. On the other hand, a difference in measurements, while initially indicative of an exclusion, must be treated with caution because of potential differences in pose angle, facial expression, camera-to-subject geometry, and other differences that may stem from the aging process or illness. Figure 1 provides an exaggerated example of how the distance between various facial features may be altered by a change in pose angle.

Figure 1: A change in pose angle results in an alteration of the distances measured between facial features. The position of the exocanthion (outer corner of the eye) is held fixed in these views, and the arrows indicate the distances from this line. The rotation of the head in the right image foreshortens the distances in the plane of the face. In contrast, the distance to the subaurale (bottom of the earlobes) has increased.

Anthroscopy is a valuable source of comparative information because there may be unusual features—such as scars, moles, or tattoos—that permit facial images to be meaningfully compared. Likewise, more common features with a presumed random distribution, such as freckles, also may permit meaningful conclusions to be drawn from comparisons. However, as with the photogrammetric approach, the visibility or appearance of some features may change as a result of aging, illness, or changes in expression. Likewise, image quality, perspective, clothing, pose angle, and artifacts of image data conversion or compression could each lead to misleading image features. Furthermore, as Farkas (1994) cautions, errors arising from anthroscopy may be greater than those from anthropometry.

Image superimposition involves the process of superimposing a known image onto the questioned image (or vice versa) and, when performed properly, should be considered a combination of anthroscopy and the photogrammetric approach. This is because, for a “match” to be achieved through superimposition, facial features and measurements must align between the questioned and known images. If such an alignment cannot be achieved, then an exclusion may be indicated. Image superimposition may be illustrative as much as investigative in value, and persistence of vision may make offender and suspect images appear more similar than they are. The assumption that pose angle has no significant effect is also made in superimposition comparisons.

Assessment of Forensic Facial Comparisons—How Can They Be Improved?

The methods of forensic facial comparisons described above closely parallel the means by which other forensic comparisons are conducted. Comparisons of latent print impressions left by human skin on the hands and feet, as well as impressions left by footwear and vehicle tires, are conducted using methods comparable to anthroscopy and image superimposition. As such, the fundamental methods employed to perform facial comparisons are sound. What is lacking, however, is a quantitative means of establishing a match between two facial images, and in the event of a match, there is no process by which to estimate the frequency of any given face shape in the general population. Not only would statistics on face shape and facial dimensions be of value, but so would statistics on the frequency and distribution of such facial features as freckles, moles, scars, and tattoos.

The development of statistics for purposes of facial identification is needed not only to improve the science but also to ensure that this type of analysis will continue to be accepted in court. Benchmark rulings relating to admissibility—in particular the “Supreme Court Trilogy” of Daubert v. Merrell Dow Pharmaceuticals, Inc., 509 U.S. 579 (1993); General Electric v. Joiner, 522 U.S. 136 (1997); and Kumho Tire Co. v. Carmichael, 526 U.S. 137 (1999); as well as Rule 702 of the Federal Rules of Evidence—require that the Court ensure that expert witnesses are reliable and that expert evidence is properly applied. While discretion in finding that evidence is relevant and reliable lies with the Court, factors that may be considered in assessing reliability include the following (e.g., Gebauer 2001): (1) the evidence is based on a theory or technique that can be, or has been, tested; (2) the theory or technique has been subject to peer review; (3) there exists a known or potential associated error rate of the theory or technique when applied; and (4) the theory or technique is generally accepted in the scientific community. The existence of controls and/or standards regarding the application of the theory or technique is another factor that is frequently considered along with the question of error rate.

Therefore, a key resource required to further scientific investigation in forensic facial comparison is a large database of precise facial measurements collected in 3-D, which can be used to develop tools for face-shape comparison and frequency estimation. Likewise, a large database of 2-D facial images can be used to develop statistics on facial features.

Here we report briefly on the development of such a database that includes both 3-D and 2-D data sets and is available for dissemination to researchers in crime prevention and detection via agreement between government agencies. The details of data collection, database contents, and structure are presented, and its value in research and arrangements for dissemination are discussed.

Facial Image Data Collection

We collected facial images of healthy volunteers at the Magna Science Adventure Centre, Rotherham, England, using digital stereophotography (Geometrix FaceVision FV802 Series Biometric Camera, ALIVE Tech, Cumming, Georgia) and 3-D laser scanning (Cyberware 3030PS Head and Neck Scanner) instruments. Volunteers, all over 14 years of age, were provided with information describing the research project and invited to complete a consent form and biographic information sheet on which age, sex, and ancestry—according to United Kingdom census classifications—were recorded. For volunteers ages 14–16 years, informed consent was additionally given by a parent or guardian.

We used a software tool (ForensicAnalyzer, ALIVE Tech) to place, with a high degree of precision, 3-D craniofacial anthropometric landmarks (Farkas 1994) on images of each face collected by stereophotography.

The Geometrix FaceVision system (Figure 2) is based on eight digital cameras, held and calibrated in a fixed geometrical alignment, that capture eight images of different aspects of the face (Figure 3). The Geometrix ForensicAnalyzer software can then be used to triangulate precisely onto any point on the face from any two camera views (Figure 4).

Figure 2: The Geometrix FaceVision system, based on eight digital cameras (black pods) calibrated and held in fixed geometrical alignment

Figure 3: An example of the eight digital photographic images of different aspects of the face captured by the Geometrix FaceVision system shown in Figure 2

Figure 4: Illustrative example of the landmarking process. A click-and-drag operation is used to place a reticule (a red or green dot) on the landmark site visible in two of the images captured by the Geometrix FaceVision system. The Geometrix ForensicAnalyzer software then triangulates onto the point (illustrated by a red or green “ray”) with precision.

A pilot study undertaken on 35 faces “landmarked” six times each at 62 landmark sites by two observers was used to evaluate inter- and intraobserver error and the effectiveness in discriminating between faces associated with each landmark. These results (not shown) were used to select a subset of 30 landmarks that showed greatest discriminating power and least associated error. This subset offered similar coverage of facial features to the initial set of 62 (see Figure 5 and Table 1). The pilot study also was used to develop a landmarking manual and induction and quality assurance procedures for landmarking technicians.

Figure 5: Illustration of the 30 landmark sites collected from 3115 volunteers using the Geometrix FaceVision equipment (see Table 1). Bilateral landmarks are in green.

Table 1: List of the 30 Landmarks Used in the Geometrix Study and Shown in Figure 5
LabelNameDescription*

g

Glabella

The most prominent midline point between the eyebrows

sl

Sublabiale

Determines the lower border of the lower lip and upper border of the chin

pg

Pogonion

The most anterior midpoint of the chin

en

Endocanthion (l, r)

The point at the inner commissure of the eye fissure

ex

Exocanthion (l, r)

The point at the outer commissure of the eye fissure

p

Pupil (l, r)

Determined when the head is in the rest position and the eye is looking straight forward

pi

Palpebrale inferius (l, r)

The lowest point in the mid-portion of the free margin of each lower eyelid

se

Sellion

The deepest landmark located in the bottom of the nasofrontal angle

prn

Pronasale

The most protruded point of the apex nasi

al

Alar (l, r)

The most lateral point on each alar contour

c’

Highest point of columella (l, r)

The point on each columella crest, level with the tip of the corresponding nostril

ls

Labiale superius

The midpoint of the upper vermillion line

li

Labiale inferius

The midpoint of the lower vermillion line

sto

Stomion

The imaginary point at the crossing of the vertical facial midline and the horizontal labial fissure between gently closed lips, with the teeth shut in the natural position

ch

Cheilion (l, r)

The point located at each labial commissure

sa

Superaurale (l, r)

The highest point on the free margin of the auricle

sba

Subaurale (l, r)

The lowest point on the free margin of the ear lobe

pa

Postaurale (l, r)

The most posterior point on the free margin of the ear

obi

Otobasion inferius (l, r)

The point of attachment of the ear lobe to the cheek

*See Farkas 1994.

Each face in the Geometrix data set was landmarked at up to 30 sites, depending on landmark visibility—some landmarks were obscured, for example, by head or facial hair. For each face, a duplicate data set was collected by independently repeating the landmarking process. The landmark data were exported to a Microsoft Office Excel 2003 spreadsheet, where each line in the spreadsheet recorded a unique key; the age, sex, and ancestry of the volunteer; and the 3-D Cartesian coordinates of each landmark in the duplicate set. Geometrix FaceVision software was also used to generate a 3-D surface for each face collected with the Geometrix scanner (see Figures 6 and 7).

Figure 6: Screen shots of the 3-D wire-framed surface of a face generated using the Geometrix FaceVision software and viewed in Internet Explorer with the Viewpoint Media Player plug-in

Figure 7: Screen shots of the texture-mapped 3-D surface of a face generated using the Geometrix FaceVision software and viewed in Internet Explorer with the Viewpoint Media Player plug-in

Contents and Structure of the Magna Database

Contents

The Geometrix data set consisted of 3115 sets of facial image data. Table 2 shows the distribution of volunteers in the Geometrix data set according to ancestry and sex. Table 3 shows the distribution by age group and sex. The Cyberware data set consists of 1844 three-dimensional faces, consisting of the geometry and a texture map.

Table 2: Distribution of Volunteers in the Geometrix Data Set
by Ancestry and Sex
Census CategoryFemalesMales
White British 1265 1553
Other White Background 56 78
White and Black Caribbean 4 3
White and Black African 1 2
White and Asian 4 6
Other Mixed Background 6 3
Indian 14 17
Pakistani 4 11
Any Other Asian Background 6 11
Caribbean 2 8
African 7 7
Other Black Background 3 1
Chinese 18 17
Any Other 4 4
Total 1394 1721


Table 3: Distribution of Volunteers in the Geometrix Data Set
by Age Group and Sex
Age GroupFemalesMales
14–19 194 188
20–24 71 68
25–29 98 103
30–34 173 188
35–39 291 325
40–44 247 361
45–49 127 181
50–54 53 111
55–59 55 65
60–64 44 54
65+ 41 77
Total 1394 1721

Structure

The Geometrix database consists of 3115 folders, each containing the raw stereophotographic image data, including eight JFIF (JPEG File Interchange Format, where JPEG is Joint Photographic Experts Group) images corresponding to the eight cameras used by the Geometrix FaceVision scanner, two XML (Extensible Markup Language) files containing the repeated sets of landmark coordinates, and the 3-D face-surface data in three standard formats—3ds (3D Studio) (Autodesk 3ds Max, San Rafael, California), DXF (Drawing Exchange Format) (Autodesk AutoCAD) and VRML (Virtual Reality Modeling Language). The uncompressed database is 213 GB in size.

The Cyberware database consists of 3-D face-surface and texture map data in Cyberware and TIFF (Tagged Image File Format) format, respectively, for 1844 volunteers. All but 144 of these volunteers were also scanned with the Geometrix scanner. The uncompressed database is 2.2 GB in size.

A separate Microsoft Office Excel 2003 spreadsheet holds the 3-D landmark coordinate data set for both repetitions of the landmark measurements for the 3115 individuals in the Geometrix database. The uncompressed spreadsheet is 5.4 MB in size.

Inter- and Intraobserver Error

All of the faces have been landmarked twice by the same or different observers, and a related investigation of the influence of error has been undertaken for Geometrix, Cyberware, and 3DMD scanners (Goodwin and Schofield 2005; Schofield and Goodwin 2006). The data set is amenable to further landmark measurement or repetition using the Geometrix FaceVision software.

Value in Research

We believe the research database of more than 3000 two- and three-dimensional facial images is the largest such database collected to date and will be a valuable resource in the investigation of face-shape variation and forensic facial comparison.

The research database offers the potential for comprehensive investigation of the frequency of facial proportions from any given pose angle and the influence of pose angle on comparisons conducted using the photogrammetric approach. Likewise, the 3-D facial surface images in the database offer the potential for the quantitative investigation of the evidential value of superimposition and consideration of the influence of pose angle in superimposition comparisons. Finally, the 3-D facial surface images in the database offer the potential for the investigation of variation in morphology of facial features and the feasibility of using statistical techniques based on the incidence of freckles, scars, moles, tattoos, and other marks in facial comparison.

We hope the database will contribute to the small but growing body of empirical research in facial comparison (see full list in Introduction), extending the quantitative and statistical resources available to the expert (cf. Saks and Koehler 2005), which can be incorporated into training, competency and proficiency standards, and best practices (SWGIT 2001, 2005; SWGDE/SWGIT 2004).

Arrangements for Dissemination

The database is available for further research in crime prevention and detection. This research is potentially law-enforcement-sensitive. Access to the database outside the United States must be arranged via agreements between government agencies. Arrangements are in place in the United Kingdom and Canada. Further details may be obtained from the authors.

Researchers are asked to respect the conditions of consent given by volunteers, who permitted their facial image data to be used in research in crime prevention and detection. Volunteers may withdraw at any time without giving a reason and have their images deleted. Facial images may not be published or otherwise disseminated. Some facial images—those of the project team—may be used, however. Further details may be obtained from the authors.

Note: Sixty individuals who volunteered to contribute to this study also volunteered to contribute to research in forensic facial reconstruction from MRI (Magnetic Resonance Imaging) (Evison and Wilkinson, manuscript under consideration). Both 3-D facial surface image and volume head-and-neck MRI image data may be available for research from these volunteers. Further details may be obtained from the authors.

For further information, please contact:

Martin Paul Evison
Department of Anthropology
University of Toronto at Mississauga
3359 Mississauga Road North
Mississauga, Ontario, Canada L5L 1C6
+ 1-905-569-4259 (Voice)
+ 1-905-569-4424 (Fax)
martin.evison@utoronto.ca
 

Richard W. Vorder Bruegge
Building 27958A, Pod E
Engineering Research Facility
Federal Bureau of Investigation
Quantico, Virginia  22135
+ 1-703-985-1192 (Voice)
+ 1-703-985-1695 (Fax)
rvorderbruegge@fbiacademy.edu
 

References

Catterick, T. Facial measurements as an aid to recognition, Forensic Science International (1992) 56:23–27.

Daubert v. Merrell Dow Pharmaceuticals, 509 U.S. 579 (1993).

Evison, M. P. Anthropometry of the face. In: Third UK National Conference on Craniofacial Identification Report on Proceedings. Department of Art in Medicine, University of Manchester, England, 2000, Abstract, p. 7.

Evison, M. P., ed. Computer Aided Forensic Facial Comparison. Unpublished Technical Report, Technical Support Working Group, Washington, D.C., 2005.

Evison, M. P., Fieller, N. R. J., Mallett, X., Schofield D., Dryden, I. L., and Solomon, C. An anthropometric approach to forensic facial comparison. Presented at the Australia and New Zealand Academy of Forensic Sciences, Fremantle, Australia, 2006.

Farkas, L. G. Anthropometry of the Head and Face. 2nd ed., Raven Press, New York, 1994.

Federal Bureau of Investigation. Laboratory examinations of photo-related evidence, FBI Law Enforcement Bulletin, May 1972, 10–15.

Gebauer, M. E. The “what” and the “how” of challenges to expert testimony under Rule 702, For the Defense (2001) 43(7):12–17, 54–55.

General Electric Co. v. Joiner, 522 U.S. 136 (1997).

Goodwin, L. and Schofield, D. Evaluation of the performance of the Cyberware, Inc. 3D scanner. In: Computer Aided Forensic Facial Comparison. M. P. Evison, ed. Unpublished technical report, Technical Support Working Group, Washington, D.C., 2005, pp. 113–144, 146–165.

Kumho Tire Co. v. Carmichael, 526 U.S.137 (1999).

Mallett, X. D. G. Evidential use of facial identification. Doctoral thesis, University of Sheffield, England, 2006.

Saks, M. J. and Koehler, J. J. The coming paradigm shift in forensic identification science, Nature (2005), 309:892–895.

Schofield D. and Goodwin, L. Facing the Future: Errors involved in Biometric Measurement of the Human Face. Presented at the Australia and New Zealand Academy of Forensic Sciences, Fremantle, Australia, 2006.

Scientific Working Group on Digital Evidence/Scientific Working Group on Imaging Technology. Guidelines and recommendations for training in digital & multimedia evidence [Online]. (October 2004). 

Scientific Working Group on Imaging Technologies. Guidelines and recommendations for training in imaging technologies in the criminal justice system, Forensic Science Communications. [Online]. (April 2002). Available: http://www.fbi.gov/hq/lab/fsc/backissu/april2002/swgittraining.htm

Scientific Working Group on Imaging Technologies. Best practices for forensic image analysis, Forensic Science Communications [Online]. (October 2005). Available: http://www.fbi.gov/hq/lab/fsc/backissu/oct2005/standards/2005_10_
standards01.htm
.

United States v. Henry Stuart Brown, No. 468, Docket 74-1947 (2d Cir. Feb. 20, 1975).

United States v. John Donald Cairns, No. 26095 (9th Cir. Nov. 6, 1970).

United States v. Tommy Louis Brown, Virgil David Swain and Robert Lee Nobles, No. 73-2279, 73-2678, 73-2280 (9th Cir. June 10, 1974).

Vanezis, P. and Brierley, C. Facial image comparison of crime suspects using video superimposition, Science and Justice (1996) 36:27–33.

Vorder Bruegge, R. W. and Musheno, T. Some cautions regarding the application of biometric analysis and computer-aided facial recognition in law enforcement. In: Proceedings of the American Defense Preparedness Association’s 12th Annual Joint Government–Industry Security Technology Symposium and Exhibition. Williamsburg, Virginia, 1996, p. 8.

Vorder Bruegge, R. W. Imaging sciences in forensics and criminology. In: Encyclopedia of Imaging Science and Technology. Vol. 1, J. P. Hornak, ed. John Wiley & Sons, Hoboken, New Jersey, 2002, pp. 709–742.

Yoshino, M., Kubota, S., Matsuda, H., Imaizumi, K., Miyasaka, S., and Seta, S. Face-to-face video superimposition using three-dimensional physiognomic analysis, Japanese Journal of Science and Technology for Identification (1996) 1:11–20.

Yoshino, M., Matsuda, H., Kubota, S., Imaizumi, K., and Miyasaka, S. Computer-assisted facial image identification system using 3D physiognomic range finder, Forensic Science International (2000) 109:225–237.

Yoshino, M., Matsuda, H., Kubota, S., Imaizumi, K. and Miyasaka, S. Computer-assisted facial image identification system, Forensic Science Communications [Online]. (January 2001). Available: http://www.fbi.gov/hq/lab/fsc/backissu/jan2001/yoshino.htm


References

This database was collected as part of a collaborative research project in computer-aided facial comparison funded by the U.S. government (TSWG T216E) and directed by the author (Evison). The authors acknowledge the contribution of the following collaborators: Professor Ian Dryden (University of Nottingham, England); Dr. Nick Fieller (University of Sheffield, England); Dr. Damian Schofield (Royal Melbourne Institute of Technology, Australia); and Dr. Chris Solomon (University of Kent at Canterbury, England); the research assistants—Dr. Gary Dickson, Lucy Morecroft, and Dr. Xanthe Mallett—and technicians who participated in the project; and the public volunteers who donated their facial images for research in crime prevention and detection. The research proposal was reviewed by a research ethics committee of the University of Sheffield, England.