Black Box Study

Test and survey results from the “Black Box” study

Test and Survey Results from the “Black Box” Study

The public data supporting the “Black Box” study1 is provided as two tab-delimited text tables described below.

Examiner responses to the test questions

The following is a description of the “Test Responses” data table containing the responses to the test questions. There is one row for each presentation of an image pair to an examiner: 169 examiners responded to approximately 100 presentations each of image pairs yielding 17,121 sets of responses. The meaning of these data values is further explained in the paper (1); see especially the instructions to participants, section 1.5 of the Supporting Information.

Column header

Data content

Data format


Unique, anonymized identifier for the examiner.  169 distinct values

(the letter “A”  followed by 3 digits)


Unique identifier for the image pair. 744 distinct values

(letter indicating mating, followed by 6 digits)
The first 3 digits uniquely identify the latent, and the last 3 digits uniquely identify the exemplar.


Whether the latent and exemplar originate from the same source



Examiner’s latent value decision



Examiner’s comparison decision

{“Exclusion”, ”Individualization, ”Inconclusive”, “NA”}


Examiner’s reason for inconclusive decision

{ “Close”, “Insufficient”, “Overlap”, “NA”}


Examiner’s reason for exclusion decision

{“Minutiae”, “Pattern”, “NA”}


Examiner’s rating of difficulty to make comparison decision

{“A_Obvious”, “B_Easy”, “C_Medium”, “D_Difficult”, “E_VeryDifficult”, “NA”}

Examiner survey responses

The “Survey Responses” has one row per survey respondent (159 participants completed the survey). The full text of each question is published in the paper (1), section 1.4 of the Supporting Information.

There is one column for each possible response to the multiple choice questions; the header of each column is of the format “QuestionNumber.Response” where Response indicates the response category; the responses selected by participants are indicated by a “Y”. For example, there are two columns for the first question, with column headers “1.Male” and “1.Female”. Some questions were of the form “check all that apply,” so multiple columns may be selected.

There is a single column for each of the free-format text responses (questions 2, 5, 7, 9, 19). No response was permitted on the free-format and “check all that apply” questions. Survey responses are not associated with Examiner_IDs to protect participant anonymity, which was part of the Institutional Review Board for Human Subject Research approval for this study (described in the paper (1), section 1.2 of the Supporting Information).

As stated in the paper (section 1.4 of the Supporting Information, #13) “On question 13, responses were available for 161 of the participants (as opposed to 159 for all of the other questions), and therefore percentages are based on a total of 161.” Two additional participants did not complete the survey but indicated that they were certified by an accredited employer.

1B.T. Ulery, R.A. Hicklin, J. Buscaglia, M.A. Roberts (2011) “Accuracy and reliability of forensic latent fingerprint decisions,” Proc. Natl. Acad. Sci. U. S. A., May 10, 2011, Vol. 108, no. 19, 7733-7738, published online before print April 25, 2011, doi:10.1073/pnas.1018707108.