“Black Box” Study Results

The public data supporting the “Black Box” study is provided below as two tab-defined text tables.1 This data discusses examiner responses to test questions and examiner survey responses.

Responses to Test Questions 

The “Test Responses” data table contains examiners’ answers to the test questions. There is one row for each presentation of an image pair to an examiner—169 examiners responded to approximately 100 presentations each of image pairs yielding 17,121 sets of responses. The meaning of these data values is further explained in “Accuracy and Reliability of Forensic Latent Fingerprint Decisions,” a 2011 research paper; see especially the instructions to participants, section 1.5 under “Supporting Information.”2

Column header
Data content
Data format
Unique, anonymized identifier for the examiner - 169 distinct values   
(the letter “A” followed by 3 digits)
Unique identifier for the image pair - 744 distinct values
(letter indicating mating, followed by 6 digits)
The first 3 digits uniquely identify the latent, and the last 3 digits uniquely identify the exemplar  
Whether the latent and exemplar originate from the same source
Examiner’s latent value decision
Examiner’s comparison decision
{“Exclusion”, ”Individualization, ”Inconclusive”, “NA”}
Examiner’s reason for inconclusive decision
{“Close”, “Insufficient”, “Overlap”, “NA”}
Examiner’s reason for exclusion decision
{“Minutiae”, “Pattern”, “NA”}
Examiner’s rating of difficulty to make comparison decision
{“A_Obvious”, “B_Easy”, “C_Medium”, “D_Difficult”, “E_VeryDifficult”, “NA”}

Survey Responses 

The “Survey Responses” data table has one row per survey respondent—159 participants completed the survey. The full text of each question is published in the 2011 research paper, section 1.4 under “Supporting Information.”3

There is one column for each possible response to the multiple choice questions; the header of each column is of the format “QuestionNumber.Response” where “Response” indicates the response category; the responses selected by participants are indicated by a “Y”. For example, there are two columns for the first question, with column headers “1.Male” and “1.Female”. Some questions were of the form “check all that apply,” so multiple columns may be selected.

There is a single column for each of the free-format text responses (questions 2, 5, 7, 9, 19). No response was permitted on the free-format and “check all that apply” questions. Survey responses are not associated with Examiner_IDs to protect participant anonymity, which was part of the Institutional Review Board for Human Subject Research approval for this study (described in the research paper, section 1.2 under “Supporting Information”).4

As stated in the paper (section 1.4 under “Supporting Information,” #13) “On question 13, responses were available for 161 of the participants (as opposed to 159 for all of the other questions), and therefore percentages are based on a total of 161.”5 Two additional participants did not complete the survey but indicated that they were certified by an accredited employer.


1 Bradford T. Ulery, R. Austin Hicklin, JoAnn Buscaglia, and Maria Antonia Roberts, “Accuracy and Reliability of Forensic Latent Fingerprint Decisions,” Proceedings of the National Academy of Sciences of the United States of America 108, no. 19 (May 10, 2011): 7733-7738, published online April 25, 2011, http://www.pnas.org/content/108/19/7733.full.
2 Ibid.
3 Ibid.
4 Ibid.

5 Ibid.