U.S. Department of Justice

Federal Bureau of Investigation

October 1999   Volume 1   Number 3

Presentations at the
2nd International Symposium on the
Forensic Examination of Questioned Documents
Albany, New York
June 14 – 18, 1999

Part 5

The following abstracts of the presentations are ordered alphabetically by authors' last names.

The Forensic Science Foundation (FSF) is the education and information arm of the American Academy of Forensic Sciences. It was founded in the 1960s to satisfy Internal Revenue Service charitable organization requirements so that donations would be tax deductible. At that time, the giants of forensic science recognized a need to establish standards for both crime laboratories and individual practitioners. The members of the Foundation are the directors of the Academy. They appoint the trustees of the Foundation often from their own numbers.

The first Executive Director of the Foundation was Ken Field, who obtained Law Enforcement Administration Act grant money to put this into practice. One of the early accomplishments of the FSF was to help set up certification boards, including the American Board of Forensic Document Examiners.

Another early goal was to establish a system of proficiency testing of forensic scientists. This had never been done, and no one had any idea how to go about it. The FSF hired a proficiency testing firm, Collaborative Testing Services, for the project.

The first test was done in 1975 and reported in an article in the Journal of Forensic Sciences by Petersen, Field, and Fabricant. That test was considered to be a dry run to test the system but not a meaningful proficiency test.

During the 1980s several proficiency tests were performed. They were considered to be operational by the FSF but with the caveat that there was a learning curve in designing the tests.

There were difficulties with many of the early tests in all areas. Some of those that particularly affected questioned documents were

• Lack of control over who took the tests. Many tested were not document examiners. A volunteer laboratory would be given the tests for all areas, even those in which that laboratory did not provide services. Some laboratories gave it a shot anyway.
  
  
• Poor materials. Questioned document tests included photographs that were not suitable for the examinations required. Some were out of focus, too high contrast, or enlarged to different magnifications without included scales.
  
  
• Lack of understanding by the test providers as to what can be deduced from physical evidence versus the known facts.

By the end of the 1980s, the American Society of Crime Laboratory Directors (ASCLD) offered laboratory accreditation. One of the requirements was proficiency testing for practitioners. The FSF ceded its proficiency testing program to ASCLD. ASCLD continues to use Collaborative Testing Services to perform the tests.

In the intervening years, the testing program has improved. To its credit, Collaborative Testing Services listened to its critics and modified the tests so that they now give an accurate indication of competency.

The FSF today is still very active. It holds an endowment out of which it provides the Lucas grants for forensic research. It also publishes needed information which is not commercially attractive to private publishers. The Foundation promotes careers in forensic science by providing brochures and sponsoring seminars.

The alteration or fabrication of a business record provides a wealth of information to the forensic document examiner and subsequently to the client. The problem to the examiner is to collect all the essential data necessary to properly evaluate the evidence. The conclusions expressed can be erroneous if not backed up by accurate and complete data. The proposed protocol has been tested on numerous cases not only by the presenting examiner but also by numerous other examiners.

This expanded protocol was the basis of the author's contribution to the American Board of Forensic Document Examiners' course on medical malpractice. The protocol is offered as the basis for a case-oriented approach to the examination of business records.

The role of the document examiner in records examinations is to locate and define any alterations found within the records. Through various forensic means, the examiner identifies the record changes and attempts to answer the questions of how, when, where, and to what extent the changes took place. The determination of the why or the intent of the alteration is more a function of the litigating parties. The intent of the alterations, however, are usually very apparent when the above questions are answered and the trends shown.

Slides of actual casework will be used in the presentation, and suggested procedural guidelines (Exhibit A) will be offered for documenting results for evaluation. The case slides should offer the viewing examiner food for thought in reviewing a great variety of cases.

Innovative approaches to evidence examination will be offered. Issues of chain of custody and presentation of results will be covered. The examination of questioned business records requires a methodical approach and fundamental but flexible procedures.

The examiner should be fully aware of the background information of the case assignment before addressing the evidence. The examiner should, however, remain objective and test the physical evidence to see if it supports the facts presented by the investigator and/or the attorney. Client bias and how it affects the examination and testimony of the examiner is also to be discussed by Howard Rile and David L. Oleksow.

The views expressed herein do not necessarily reflect those of the United States Department of Justice.

Introduction

This talk is neither an official Department of Justice (DOJ) position nor an attempt at scholarship. It is just the thoughts of one former prosecutor who has worked with questioned document evidence in the post-Daubert world and has seen how questioned document examination (QDE) experts and prosecutors can and sometimes must adapt to the changing case law in order to get the evidence before jurors.

The advent of Daubert¹ changed the law of expert testimony admissibility from the Frye² test that formerly gave prosecutors distinct advantages in a criminal trial. This change might at first have been expected to be the kind that prosecutors would not find objectionable: opening the door to the admission of novel types of expert testimony that could be tested and, if appropriate, refuted through vigorous cross-examination. However, particularly in the field of handwriting identification, the combined effect of Daubert and recent criticisms of QDE experts has been both unexpected and disturbing for prosecutors: the restriction or even outright exclusion of opinion testimony by QDE experts on the grounds that their testimony cannot be considered reliable under the four-part reliability test announced in Daubert.

Although these recent developments are troubling, prosecutors and QDE experts can work together to ensure that the results of the experts' valid and valuable analysis is presented to jurors in a form that allows the prosecutor to achieve his or her task: proving that the allegations in an indictment are true beyond a reasonable doubt. In the post-Daubert era, this requires greater commitment by the prosecutor to learn the theories and methods that QDE experts bring to their tasks. It also requires prosecutors and QDE experts to work together to develop alternative strategies for communicating the results of the expert's analysis to the jury in cases such as the Oklahoma City bombing trial where the traditional method of having the expert attest to an identification opinion is precluded under Daubert. Finally, and particularly in light of the recent decision in Kumho Tire³, Daubert challenges prosecutors and the QDE community to work with scholars to develop ways to demonstrate to courts either that the basic principles of QDE analysis are scientifically valid or that, as a statistically significant empirical matter, QDE experts are better able than lay persons to identify the authors of questioned documents.

Pre-Daubert Regime

Before Daubert, the admissibility of expert testimony was governed by the Frye general acceptance test. For all of its flaws, Frye offered very real advantages for the prosecutor, both in terms of what it admitted and what it excluded. Most of the forensic tools upon which prosecutors tend to rely easily passed the Frye test: document comparison, toolmark analysis, expert testimony about organized crime, and many others. By contrast, the Frye test tended to exclude types of testimony that prosecutors rarely if ever need and frequently oppose: novel scientific or sociological theories intended to cast doubt on what a prosecutor would likely characterize as a common-sense theory of the case and, in particular, polygraph evidence.

In a rather oversimplified sense, the Frye test almost inevitably favored the prosecutor. The government tends to bring a lot of cases that it tries to prove using a relatively limited set of tools; accordingly, the types of expert testimony on which prosecutors rely are more likely to lead to the development of a community of experts who generally accept each others' methodology. Indeed, that is essentially one of the criticisms of QDE in the University of Pennsylvania Law Review⁴ article: That document examiners as a group passed muster under the Frye test by accepting certain basic assumptions and methods despite having failed to justify them through scientifically acceptable validation testing.

But a defendant seeking to undermine a strong case against him may have an incentive to seek out and rely upon novel theories precisely because they are novel. Other, more accepted and predictable defenses may appear unlikely to succeed, and the defendant may have reason to hope that the prosecution will not be prepared to refute a novel theory. However, the very novelty that may make a theory attractive in terms of defense strategy is exactly what made admission of the evidence less likely under the Frye test, regardless of how scientifically valid the new theory could be shown to be.

"Good Stuff" and "Bad Stuff"

In short, and in far too simplistic terms, the general acceptance test of Frye let the "good stuff" in and kept the "bad stuff" out. These terms are not intended to describe scientific validity, but rather the usefulness to the prosecutor's task of proving allegations beyond a reasonable doubt. And QDE evidence, from that perspective, is plainly part of the "good stuff." Notwithstanding the fact that the best examiners testify for both prosecutors and defendants (as should be true of experts in any field), the admissibility of QDE evidence is enormously valuable to a prosecutor because it is a discipline that tries (perhaps too eagerly, in the view of its critics) to provide answers rather than to cast doubt. By contrast the "bad stuff" includes disciplines or techniques that are used primarily to undermine the proposition that the truth can ever be known with any acceptable degree of certainty, which I think is generally the case, for example, with experts in the reliability of eyewitness identification or with polygraph evidence. It is worth adding that although I do not intend "bad stuff" to mean the same thing as "junk science" (a term some critics unfairly apply to QDE), I doubt that a responsible prosecutor should ever be opposed to the admission in court of scientifically valid expert evidence, even if the ultimate conclusion suggested by the evidence is that some questions cannot be answered.

Daubert Opened the Gate

When Daubert was decided in 1993, it initially seemed like the primary effect of the decision would be on only one side of that formulation: More of the "bad stuff" might be admitted, but it did not appear to endanger the admissibility of the "good stuff." That is because the most obvious criticism of the Frye standard was that it was not letting in valid scientific developments that were simply new.

To a very limited extent, that was bad news. Of particular concern to prosecutors was polygraph evidence, which, after Daubert, appeared more likely to be admissible. What is the difference between polygraph, the admissibility of which prosecutors generally oppose, and QDE evidence, which prosecutors favor? From my perspective, it is that polygraph machines use unreliable proxies in an attempt to establish too slippery a concept of only indirect relevance to a case (e.g., the witness's deceptiveness as to a given question rather than the actual truth or falsity of a factual proposition). By contrast, QDE experts use transparent methods in an attempt to establish a specific proposition of direct relevance (e.g., whether a specific person wrote a specific document). However, both share a common vulnerability, in that each relies on an unproven axiom: polygraphy's being that certain autonomic responses are indicative of deception, and QDEs being that each individual's handwriting is demonstrably distinct.

But in a more basic sense, if all that Daubert had done was to open the courtroom doors wider, it would not have had a great impact on the way prosecutors do business. The prosecutor is an advocate who must ensure that his evidence can withstand the scrutiny of an adversary and who is in turn prepared to meet and refute any evidence presented by a defendant through cross-examination and rebuttal evidence. As long as the prosecutor remains free to use traditional tools to put on his case, the admission of additional defense expert evidence under Daubert might make the prosecutor's job somewhat more challenging, but only as a matter of degree rather than of kind.

To be sure, the Daubert opinion referred to the trial judge's role as gatekeeper. But, at least in the early days after Daubert, prosecutors primarily saw that as an admonition to be more liberal in evidentiary rulings by keeping the gate open to disciplines that had formerly been excluded. It did not seem to be an invitation to close that gate to those that had always been allowed in.

Post-Daubert Attacks on Handwriting

Prelude: The University of Pennsylvania Law Review Article
Even before Daubert, the 1989 article in the University of Pennsylvania Law Review by Professors Risinger, Denbeaux, and Saks started raising serious questions about the scientific validity of QDE evidence. Whatever the critics and defenders of the article may say—and they have said a lot in the intervening decade—at the very least it must be conceded that the article's authors caught the attention of examiners in the field, scholars, judges, and attorneys and forced all of us to think seriously about the validity of QDE evidence and the reasons why it does or does not belong in a courtroom.

For the most part, I count myself among the article's critics, although I readily concede that, as a prosecutor who has tried to rely on handwriting identification evidence, I am biased. And although many criticisms of the article are not particularly relevant to the question of whether QDE evidence should be admissible in federal court after Daubert, some criticisms do highlight the ways in which the QDE community can and should try to adapt to Daubert.

Specifically, I think the article unfairly criticized the QDE community for failing to develop a rigorous empirical defense of its theories and methods. Like many forensic disciplines and unlike other scientific fields that can support research and marketing outside the courtroom, forensic document examiners traditionally had not had any particular reason to conduct validity studies because their testimony was being admitted without them. This may well be one of the valid criticisms of the Frye regime, but it is not a basis for arguing that QDE evidence is unreliable.

Second—and this is not a criticism of the article's fairness, but rather a disagreement with its conclusions—the article essentially (if not explicitly) denies what we know intuitively to be true and what the law continues to recognize: Some persons who did not see a handwritten document being written can nevertheless identify its author by visual inspection of the writing. The current legal debate is not over that basic proposition. Family members and friends are plainly allowed to say conclusively that they recognize a writing as having been written by a particular person with whose handwriting they have become familiar⁵. Rather the debate is over whether the ability to recognize a particular person's handwriting can be achieved through the methods used by QDE experts—methods which essentially duplicate the methods by which a relative or friend gain such familiarity. Stated that way, it is surprising that there should be a valid objection to the admission of expert QDE testimony under Daubert.

Despite such criticisms, the University of Pennsylvania Law Review article does make some valid points about the usefulness of statistical validation studies. If what we know intuitively and what the law presumes is true, then it can not hurt to conduct such studies and, in fact, can only serve to quell criticism of QDE analysis—provided the critics are open to positive results of such studies as well as negative.

Post-Daubert Cases Start Putting the University of Pennsylvania Law Review Article Into Practice
Until Daubert, the University of Pennsylvania Law Review article did not have an appreciable effect in the courtroom on the admissibility of QDE evidence. However, starting in 1995, the criticisms expressed in the article started to affect courtroom practice.

In Starzecpyzel⁶, the district court held a lengthy hearing to determine whether QDE expert testimony is admissible as valid science under Daubert. The court ultimately found that QDE is not a science but admitted the expert testimony (with important limitations on the gradations of confidence in an association that the expert could use) on the theory that QDE is a field of technical expertise, and that a handwriting expert's testimony is comparable to that of a harbor pilot's.

Starzecpyzel, and similar decisions that came later, may have troubled examiners who prefer to think of QDE as a science, but it did not, at least directly, signal that the courtroom had closed its doors to their testimony. For the prosecutor, the case had a somewhat different meaning: The price of introducing expert QDE testimony had gone up and could henceforth require a long and difficult pretrial hearing into admissibility.

Part of the problem revealed by Starzecpyzel was that the Daubert opinion had been written in a way that allowed opponents of handwriting analysis to put the discipline in a box: choosing between calling it a science (and defending it as such) or calling it nonscientific, technical expertise and arguing that Daubert simply did not apply. The latter course proved safer, but in some ways antithetical to the way QDEs have been doing business. As discussed below, Kumho Tire, disappointing as it may have been to examiners for not saying that Daubert does not apply, at least rebutted the false dichotomy by saying that Daubert does apply, but that its four factors Cbeing merely illustrativeCare not inflexibly applied in all cases. Thus, although the district court in Starzecpyzel wrote that Daubert does not apply to nonscientific expert testimony—a proposition rejected this year in Kumho Tire—the court's reasoning would likely survive scrutiny after Kumho Tire, in that it looked for indicia of reliability outside the rigid boundaries of the four Daubert factors.

Oklahoma City Bombing Cases
McVeigh⁷ proved that Starzecpyzel was not just an aberration. In an extremely important and well-publicized criminal case, a trial judge all but excluded QDE opinion evidence. The judge in McVeigh further rejected the argument that regardless of its effect on other fields of expertise under Rule 702, the Daubert opinion did not affect the admissibility of testimony by a handwriting expert to authenticate a document under the Federal Rule of Evidence 901(b)(3).

The court also went further than the judge in Starzecpyzel by forbidding all opinion testimony except an opinion as to the visual similarity of particular characteristics of known and questioned documents (e.g., this letter "t" in the known sample looks similar to this letter "t" in the questioned document). In essence, the court ruled that a QDE examiner is only an expert to the extent that his experience has trained him to know what to look for but is not permitted to give an opinion as to the authorship of a questioned document.

Adapting to Daubert

As should be apparent, the ruling on handwriting evidence in McVeigh gave the prosecutors in that case, myself included, ample opportunity to think about how to adapt, as a practical matter, to Daubert and its progeny. Here are some of the lessons I took away from my experiences in that case.

Prosecutors Have to Learn to be Experts
QDE examiners spend their professional careers learning how to become experts in the field, honing their skills and applying them in case after case. Prosecutors, of course, cannot and need not try to match that expertise. But, especially after Daubert, a prosecutor seeking to rely on document analysis for the identification of a questioned document's author can no longer simply wait for the expert to arrive in his office for a short preparatory session armed with the standard list of questions and answers. The prosecutor may well have to prepare for a Starzecpyzel-type hearing that explores the theoretical and practical underpinnings of QDE analysis and at the very least must be prepared to explain to the judge why no such hearing is needed. In presenting the testimony, the prosecutor may well have to abide by restrictions on the expert's testimony, and such restrictions may vary from one courtroom to another. The prosecutor who does not learn the basic principles of QDE may not be able to adapt to such restrictions and will have a hard time preparing the expert to get as much evidence in as the court will allow.

In McVeigh, where we were blessed with greater resources than are normally available to a prosecutor, I was able to take the time to learn about QDE. I studied Ordway Hilton's book⁸ and read some of the leading articles in the field, including the University of Pennsylvania Law Review article, before I first met with our expert witness. And once I did meet him, I invested a lot of time and effort to find out not just what he concluded and how to elicit the testimony, but how the analysis was done and how it might be challenged.

To some extent, this is just plain common sense, regardless of Daubert. It is also a result, to some extent, of the addition of the Federal Rule of Criminal Procedure 16(a)(1)(E), also in 1993, which required the disclosure of summary and bases or reasons for the expert's testimony. We did a very extensive bases/reasons letter in McVeigh, and although it was a lot of work, it forced me to learn the proof well enough so that when we ultimately decided not to present the expert testimony under the restraints imposed by the court, we nevertheless knew what to use, if necessary, in argument.

Prosecutors Must Work With Experts to Teach Jurors to be Experts
The flip side of the greater need for the prosecutor to learn about what QDE experts do and how they do it, is an expanded role for the expert beyond doing the analysis and reporting his or her conclusions in court by means of a standard question and answer. In the face of restrictions that prevent experts from reporting some or all of their conclusions as opinion testimony, prosecutors and experts must find a way to teach the jurors to arrive at the same conclusions themselves.

Ultimately, it is the jurors who must decide whether the expert's conclusion is correct. If the jurors can be taught how to check the expert's work for themselves (and, of course, if the expert's conclusion is in fact correct), that is the conclusion they will reach regardless of whether the expert is allowed to state his conclusion in straightforward terms. In addition to being a practical necessity in some cases (like McVeigh and, to a lesser extent, Starzecpyzel), this is also likely to improve the persuasiveness of the expert testimony even where the examiner is permitted to give the traditional kind of opinion testimony.

Continue to Press the Argument Under Rule 901
In McVeigh, the judge rejected the argument that Rule 901(3) permits expert QDE testimony after asking whether any case relied on that Rule in holding that such expert testimony was admissible despite Daubert. Shortly after that, the Jones⁹ decision in the Sixth Circuit did just that.

Potential Lessons for Examiners

As discussed previously, Daubert has had the unfortunate consequence in some cases of forcing QDE experts to fit their discipline into a box labeled science or another labeled technical expertise, when neither description may accurately fit the work that QDE examiners do. However, to the extent that such a choice has been forced on the field—and in light of this year's Kumho Tire decision, the choice may be less pressing—there are some things that practitioners can do.

If You Want To Be Scientists, Do Science
One valid criticism—though perhaps one that is aimed at a straw man—is that QDE experts should not hold themselves out as scientists without the support of experimental results based on the scientific method. For example, as I understand it, QDE examiners operate on the basis of two axioms: No two persons write alike, and no individual ever reproduces a writing in exactly the same way twice. If expert testimony in this field is to be offered as scientific knowledge, it is fair to expect that those assumptions can be tested and proved through scientific experimentation. Such tests are undoubtedly difficult to conduct, but they are essential if the label of science is to be applied to this field. Likewise, there has to be some convention as to the meaning of terms and gradations of associations. Otherwise, expert testimony in the field will rightly be subject to the criticism that forms of association such as probable are effectively meaningless.

If You Want To Be Specialized Technicians, Prove That You Can Do It
QDE testimony need not be characterized as science to be admissible, as was shown in Starzecpyzel. And if offered and admitted as specialized technical knowledge, or knowledge based on experience, it is less important to prove the axioms mentioned above than it is to prove that the expert can—for whatever reason—be successful in identifying questioned documents through certain forms of examination. This is where the research that Professor Kam has been doing at Drexel University^10,11 can be quite useful and should be pursued. Although critics of QDE, such as the authors of the University of Pennsylvania Law Review article, may have been too reluctant to accept Professor Kam's results—and it is equally important for such critics to be fair in their assessment of such studies as it is for proponents of QDE not to place more reliance on them than they warrant—their criticism should be seen as a useful guide for further study.

Ultimately, on the basis of the results of Professor Kam's studies, it seems likely to me that experiments can prove as an empirical matter what many people believe at least intuitively: namely, that reliable handwriting identification is possible and that examiners with certain types of training and practice are significantly more likely to be more successful at it than lay jurors. Particularly after the decision in Kumho Tire, that should be a sufficient basis for admitting QDE testimony in future cases.

After Daubert

Kumho Tire, decided this spring, mixes good news and bad news for QDE experts and prosecutors seeking to introduce their testimony. The most obvious bad news is the holding that the basic principles of Daubert—requiring the trial judge to assume a gatekeeping function to guard against evidence deemed unreliable—are not limited to scientific expertise but instead govern all forms of expert testimony offered under the Federal Rule of Evidence 702. Further, the language of the opinion makes clear that challenges to the entire discipline of QDE analysis, as opposed to the qualifications or methods of individual examiners, will continue to be raised and, perhaps, accepted in future cases.

On the other hand, the good news is that the Court emphasized that the four Daubert factors need not be applied rigidly in every case and that they are instead illustrative guidelines for a trial judge performing the gatekeeping function. This is an important point, and one that may make the difference between cases such as Starzecpyzel, where the court used essentially similar reasoning to allow expert opinion testimony on the ultimate issue of identification, and McVeigh, where such testimony was not allowed because it was deemed not to qualify as science.

Kumho Tire provides an added incentive to pursue research like that begun by Professor Kam. As long as there is some reasonable basis for the judge to open the gate, it should be opened—even if QDE does not meet the stringent, and in many ways inapposite, tests of science.

In particular, Kumho Tire gives prosecutors and QDEs an opportunity to work together (though the lead will likely have to come from the QDE community) to develop persuasive explanations of why such expert testimony is reliable—explanations that, after Kumho Tire need not be based exclusively, or even at all, on the four Daubert factors. However, it is probably worth pausing to consider that those four factors were identified in Daubert because, at least with respect to science but not exclusively so, they are indicia of reliability—and the practitioners of any discipline who disregard them entirely do so at the risk of their own credibility.

But if you are going to be harbor pilots rather than scientists, use the language of harbor pilots. Do not try to distinguish between probable and certain without being able to explain the difference in defensible terms or without doing the kind of validation study that Professor Kam has done with respect to identifications as such. Kumho Tire is an opportunity for both prosecutors and examiners to regain some of the traditional advantages that have been scaled back in the wake of Daubert. But we cannot take advantage of that opportunity unless we learn and take to heart some of the valuable lessons of the Daubert years: Be flexible, and be willing and able to prove to judges and juries in each case that you can do what you say.

References

1. Daubert v. Merrell Dow Pharmaceuticals, Inc., 509 U.S. 579 (1993).

2. Frye v. United States, 293 F. 1013, 1014 (D.C. Cir. 1923).

3. Kumho Tire Co., Ltd. v. Carmichael, 119 S.Ct. 1167 (1999).

4. Risinger, R. M., Denbeaux, M. P., and Saks, M. J. Exorcism of ignorance as a proxy for rational knowledge: The lessons of handwriting identification expertise, University of Pennsylvania Law Review (1989) 137:731-792.

5. See, e.g., Federal Rule of Evidence 901(b)(2) (permitting authentication by means of "[n]on-expert opinion as to the genuiness of handwriting, based on familiarity not acquired for purposes of the litigation").

6. United States v. Starzecpyzel, 880 F.Supp. 1027, 1036 (S.D.N.Y.1995).

7. United States v. McVeigh and Nichols, 98-CR-68M, Transcript of Pre-Trial Hearing, 1997 WL 47724 (D.Colo. Feb, 5, 1997).

8. Hilton, O. Scientific Examination of Questioned Documents. CRC Press, Boca Raton, Florida, 1993.

9. United States v. Jones, 107 F. 3d 1147, 1159-60 (6th Cir.), cert. denied, 571. U.S. 1127 (1997).

10. Kam, M., Wetstein, J., and Conn, R. Proficiency of professional document examiners in writer identification, Journal of Forensic Sciences (1994) 39:5-14.

11. Kam, M., Fielding, G., and Conn, R. Writer identification by professional document examiners, Journal of Forensic Sciences (1997) 42:778-786.

Back to index

FORENSIC SCIENCE COMMUNICATIONS     OCTOBER 1999   VOLUME 1   NUMBER 3