|
|
|
July 1999 Volume 1 Number
2
Presentations at the
International Symposium on Setting Quality Standards for the
Forensic Community
San Antonio, Texas
May 3-7, 1999
Part 2
The following abstracts
of the presentations are ordered alphabetically by authors' last
names.
Scientific
Working Group on DNA Analysis Methods (SWGDAM)
B. Budowle
Federal Bureau of
Investigation
Quantico, Virginia
DNA typing has been a boom
to forensic science. It has enabled the resolution of cases where
not previously possible, has been used to exonerate individuals
falsely associated with evidence, and has provided numerous investigative
leads. The full use of DNA typing technology (i.e., development
of high quality, sensitive, and reliable typing methods and the
establishment of a national DNA data bank) only could have come
to fruition by coordinating the resources of the forensic laboratories
in North America. The FBI Laboratory, in concert with several
North American forensic science laboratories, recognized the
need for setting peer-consensus standards of performance and
creating an infrastructure of communication and compatible data,
with the ultimate goal being the development of a national DNA
data bank called CODIS (Combined DNA Index System). To achieve
this end, the Technical Working Group on DNA Analysis Methods
(TWGDAM) was created in 1988. TWGDAM was the first working group
in the North American forensic science community and is hosted
and supported by the FBI Laboratory. Recently, the working group
has been renamed the Scientific Working Group on DNA Analysis
Methods (SWGDAM) to better reflect its mission. More than 30
laboratories send members to participate in SWGDAM. Also, a community-wide
meeting is held annually to foster communication and promote
greater input.
The peer-consensus approach
has established many guidelines and scientific studies regarding
forensic DNA typing. Areas that have been addressed by SWGDAM
include
- RFLP typing compatibility,
- chemiluminescent detection
of RFLP profiles,
- quality control/quality
assurance guidelines,
- ASCLD/LAB
accreditation issues,
- audit guidelines,
- statistical interpretation
issues,
- CODIS criteria,
- collection of population
data,
- criteria for PCR-based typing,
and
- criteria for mitochondrial
DNA typing.
Many of the products of SWGDAM
efforts are recognized worldwide. In fact, the DNA Identification
Act of 1994 contained within the Omnibus Crime Control
Act (The Violent Crime Control and Law Enforcement Act
of 1994, Public Law 103-322, 108 statute 1796) recognized
TWGDAM Quality Assurance Guidelines as de facto standards for
forensic DNA typing laboratories. However, the Act also
authorized the FBI Director to establish quality assurance standards
for laboratories performing forensic DNA testing through recommendations
of a DNA Advisory Board (DAB). Thus, SWGDAM took on a unique
role for a working group and volunteered to support the DNA Advisory
Board in its effort. In essence, SWGDAM became a working arm
of the DAB. Once the mission of the DAB ends (March 2000), SWGDAM
will be the entity that makes recommendations to the FBI Director
for changes to the quality assurance standards for forensic DNA
typing.
The mission of SWGDAM will
continue to be to
- bring together organizations
actively pursuing various forensic DNA analysis methods for the
purpose of exchanging and disseminating information;
- discuss, share, and compare
forensic DNA analysis methods, protocols, and research;
- establish forensic DNA analysis
quality assurance guidelines and quality controls;
- cooperate with other national
and international organizations in developing standards for forensic
DNA analyses;
- serve as a mechanism for
the review and revision of guidelines and standards for forensic
DNA analyses; and
- disseminate to the forensic
DNA community SWGDAM's guidelines, forensic research results,
and other work products of benefit to the forensic DNA community.
Current work of SWGDAM focuses
on training guidelines, STR interpretation guidelines, and validation
criteria.
Back to index
Statistics
G. Carmody
Carleton University
Ottawa, Ontario, Canada
Statistics is often thought
to be an unnecessary frill if a scientific experiment has been
done correctly. This idea is a misconception. Analysis of experimental
error is at the very core of the scientific method. In the following,
I give examples of how, starting with asking what the problem
or question is, statistics can be used to formulate an answer.
Once a question is framed,
simple decisions about how to organize and display the data in
a fashion that exposes its relevant structure can be made. In
this presentation, I have explained the difference in the terms
accuracy, validity, precision, reliability, bias, sensitivity,
and specificity. I also have presented the standard Neyman-Pearson
scheme of hypothesis testing and show how it applies to diagnostic
testing, as well as accepting or rejecting an hypothesis.
I discuss two examples from
the quality control literature where the absence of data can,
paradoxically, be useful in uncovering an important source of
error for quality control. I also describe the phenomenon called
Benford's Law, which allows us to use the surprisingly nonrandom,
yet predictable, pattern of numbers to detect fraudulent tax
returns.
Back to index
Scientific
Working Group for the Analysis of Drug Samples (SWGDRUG)
A. B. Clark
Drug Enforcement
Administration
Dallas, Texas
The Scientific Working Group
for the Analysis of Drug Samples (SWGDRUG) has met three times
since 1997. The overall mission of SWGDRUG is to make recommendations
for internationally accepted minimum standards for the forensic
examination of seized drugs. The SWGDRUG Conference is an international
endeavor that features representative managers and practitioners
from forensic science organizations in the United Nations, United
States, Canada, United Kingdom, Japan, Germany, and Australia.
Organizationally, the SWGDRUG core committee is composed of four
subcommittees having distinct missions, as follows:
- Communications: The mission
of this subcommittee is to convey to the international forensic
science committee the establishment of SWGDRUGCits
goals and objectivesCand to promote the adoption of and
adherence to the recommended standards.
- Methods and Reports: This
subcommittee is to assess and evaluate available analytical methodologies
and reporting methods.
- Quality Assurance: The mission
of this subcommittee is to develop quality assurance guidelines
for the operation of a forensic drug analysis laboratory.
- Education and Training:
The mission of this subcommittee is to recommend minimum education
and training requirements, competency standards, assessment protocols,
and guidelines for continuing professional development for forensic
drug examiners.
Representatives from the
four subcommittees made reports and proposals that reflect their
commitment to the process:
- The Communications Subcommittee
assigned core committee members to present a report of the conference
at regional, national, and international forensic science meetings.
- The Methods and Reports
Subcommittee discussed with the core committee the survey results
and their plans to formulate proposals based on the facts contained
therein.
- The Quality Assurance Subcommittee
made a number of proposals, which were published in the March
1999 issue of Microgram.
- The Education and Training
Subcommittee made the first set of proposals, which were disseminated
in the December 1998 issue of Microgram.
As more information becomes
available, it will be disseminated on the SWGDRUG World Wide
Web site (http://www.erols.com/scitechz)
and in Microgram. Everyone is invited to participate in
the process by sending their comments to SWGDRUG core committee
members. The list of members, with contact information, is available
on the SWGDRUG web site.
Back to index
Qualifications of a Scientific Expert Witness
S. J. Cribari
University of Denver College of Law
Denver, Colorado
In discussing the law governing
the qualifications of an expert scientific witness, I offer insights
on the strategy and tactics of preparing yourself and your material
for direct and cross-examination in court, as well as on scientific
honesty and professional ethics in an adversarial system of justice.
Back to index
Statistical Process Control or Graphical Quality
Evaluation, Demonstration, and (Maybe) Control
D. L. Duewer
National Institute
of Standards and Technology
Gaithersburg, Maryland
How do you convince yourself
that results provided by some measurement system are trustworthy?
How can you tell when the system is starting to act up before,
you hope, it savages you? If the system is acting up, how do
you identify what has changed? How can you convince yourself,
your management, or both that you know what you're talking about?
Although the term statistical
quality control may suggest esoteric technology, the answers
to the previously listed challenges can be accomplished graphically.
Control chart techniques are sufficiently common sense to be
interpreted by nonspecialists yet are powerful and rigorous.
The basic tool is the simple scatterplot, plotting measurement
results versus time. With a little attention paid to the scale
of both axes, these simple charts simultaneously document how
stable the measurement system has been, whether the system is
changing, and where the system may be going. Superimposing various
measurement quality indicators (whether set by regulation, policy,
or statistical analysis) documents how good the measurement system
is relative to how good it needs to be.
This presentation has been
drawn from National Institute of Standards and Technology's experience
with forensic RFLP measurement systems. I have focused on what
is appropriate for getting where you need to go:
- What is an appropriately
quantitative measurement?
- What is an appropriate suite
of blank, check, control, or standard samples?
- What are appropriate graphical
tools?
The hard requirements—taking
the data and getting it stored in a useful form—are probably
already part of your routine protocol. If they're not, they should
be! The only added requirements for successful evaluation, demonstration,
and control of measurement quality are management support, personal
dedication, and a trained intuition. The latter is facilitated
by always remembering to DTDP-ATLAI (Draw the Darned Plot, And
Then Look At It!) and KISS (Keep It Short and Simple).
Many different computer programs,
short courses, workshops, books, CD-ROM self-instruction kits,
applications, and diatribes can be found on the World Wide Web
using the search phase statistical process control. I recommend
the advanced search at
http://www.altavista.com
The NASA Headquarter's Library
maintains a very useful list of statistical process control references
at
http://www.hq.nasa.gov/office/hqlibrary/ppm/ppm31.htm
Back to index
DNA Advisory Board Standards
A. J. Eisenberg
University of North Texas Health Science Center
Fort Worth, Texas
The DNA Advisory Board (the
Board or DAB) was established by the Director of the Federal
Bureau of Investigation under the DNA Identification Act of
1994 (the Act) as a separate and distinct advisory
board administered by the FBI. The Act provided funding
for forensic laboratories to improve the quality and availability
of DNA analyses and for the FBI to establish the national Combined
DNA Index System (CODIS). The U.S. Congress authorized the Director
of the FBI to develop standards to assure quality—including
standards for testing the proficiency of forensic laboratories—and
to provide guidance to forensic analysts performing DNA analyses.
The legislation specified that access to federal funding provided
under the DNA Identification Act is contingent upon a
laboratory's strict adherence to these standards. In accordance
with the DNA Identification Act, the quality assurance
guidelines adopted by the Technical Working Group on DNA Analysis
Methods (TWGDAM) were deemed the FBI Director's interim standards.
Members of the Board were
appointed by the Director of the FBI from nominations proposed
by the National
Academy of Sciences and professional societies of crime laboratory
officials. The Act specified that members of the Board
include scientists from state, local, and private forensic laboratories;
molecular geneticists and population geneticists not affiliated
with a forensic laboratory; a representative from the National
Institute of Standards and Technology (NIST); the chair of
the Technical Working Group on DNA Analysis Methods (TWGDAM);
and a judge. Initially, the Board consisted of 16 members, including
the chair (Nobel prize winner Dr. Joshua Lederberg, a voting
member); two voting members in a representational capacity (from
NIST and TWGDAM); ten general voting members; and three nonvoting
members. The three nonvoting members appointed by the FBI Director
included an executive secretary, a quality control/quality assurance
specialist, and a medical/legal ethicist. In addition, pursuant
to the Federal Advisory Committee Act, the Director of
the FBI appointed a designated federal employee to perform the
functions set forth by law.
The Director of the FBI,
in accordance with the DNA Identification Act of 1994,
specified that the Board's scope of activity was to
- develop, recommend, and
if appropriate, periodically revise standards for quality assurance,
including standards for testing the proficiency of forensic laboratories
and forensic analysts, in conducting analysis of DNA;
- recommend standards that
specify criteria for quality assurance and proficiency tests
to be applied to the various types of DNA analyses used by forensic
laboratories, including statistical and population genetics issues
affecting the evaluation of the frequency or occurrence of DNA
profiles calculated from pertinent population databases;
- recommend standards for
acceptance of DNA profiles in the FBI's Combined DNA Index System
(CODIS) that take into account relevant privacy, law enforcement,
and technical issues; and
- make recommendations for
a system for grading proficiency-testing performance to determine
whether a laboratory is performing acceptably.
Throughout its deliberation
concerning these quality standards, the DNA Advisory Board recognized
the need for a mechanism to ensure compliance with the standards.
An underlying premise for these discussions was that accreditation
would be required to demonstrate compliance with the standards
and, therefore, assure quality control and a quality program.
The Board recommended that forensic laboratories performing DNA
analysis seek such accreditation with all deliberate speed. The
Board strongly encouraged the accrediting bodies to begin positioning
themselves to accommodate the increasing demand for accreditation.
In accordance with the DNA Identification Act, the Board
forwarded its recommendations for standards to the Director of
the FBI. After careful review, the Director's Quality Assurance
Standards for Forensic DNA Testing Laboratories took effect on
October 1, 1998. These national standards now have superseded
the TWGDAM Guidelines for purposes of certifications required
for the receipt of federal funding as well as participation in
the National DNA Index System.
Inherent within the standards
is the intent to upgrade the educational background and experience
of forensic scientists. The Board spent a considerable amount
of time discussing the qualifications for the technical manager/leader
(or equivalent position or title as designated by the laboratory
system). This is the individual who is accountable for the technical
operations of the laboratory. The Board expressed strong belief
that this position required an individual with a minimum of a
master's degree in biology, chemistry, or a forensic science-related
area. In addition, the technical manager/leader must have successfully
completed a minimum of 12 semester credit hours (or equivalent)
in course work covering biochemistry, genetics, and molecular
biology or other subjects that provide a basic understanding
of the foundations of forensic DNA analysis as well as statistics,
population genetics, or both as applied to forensic DNA analysis.
A technical manager/leader of a laboratory must have a minimum
of three years of forensic DNA laboratory experience. The Board
recognized that there are individuals who have been serving in
the capacity of technical manager or leader who do not meet the
educational requirements specified by the standards. The standards
include a provision for the waiver of the degree requirements
for the technical manager/leader. The waiver is only for the
educational requirement and does not apply to the minimum of
three years of forensic DNA laboratory experience. ASCLD, in
conjunction with ASCLD/LAB, has established a credentials review
committee to grant waivers to current technical leaders/managers
who do not fully meet the DAB educational standards for the DNA
technical leader/manager position, but who qualify as technical
leaders on the basis of their knowledge and experience. The waiver
is available for a period of two years from the effective date
of the standards (October 1, 1998). In addition, the wavier is
permanent and portable.
Forensic DNA identification
analysis currently involves forensic casework and convicted offender
analyses for inclusion in CODIS. Separate standards have been
drafted for laboratories performing the analysis of convicted
offender samples. The separation was an acknowledgment of the
differences in the nature or type of sample, the quantity and
potential for reanalysis, and specialization that may exist in
a laboratory. Standards for convicted offender laboratories,
in some instances, are less stringent than for those laboratories
performing forensic casework analyses, but in no case should
the two documents be interpreted as conflicting. The Board forwarded
its recommendations to the Director of the FBI. Following review,
the Director issued his Quality Assurance Standards For Convicted
Offender DNA Databasing Laboratories. These standards became
effective April 1, 1999.
Pursuant to Section 210303
of the DNA Identification Act of 1994, the DNA Advisory
Board was tasked with recommending standards for testing the
proficiency of forensic laboratories that conduct analyses of
DNA. The Act required that the National Institute of Justice
(NIJ) review the availability and feasibility of blind external
proficiency testing for forensic DNA analysis. A grant was awarded
by NIJ, and after an extensive evaluation and review, the project's
participants concluded that by defining feasible as possible
and practicable in terms of costs and logistics, a national blind
proficiency-testing program employing blind proficiency tests
administered by law enforcement agencies, conduit laboratories,
blind analyst models, or a combination of these is not feasible
at this time. Three recommendations were provided in their report:
"1. The accreditation
system and associated quality assurance guidelines of the DNA
Advisory Board need to be given the opportunity to take hold,
2. It is recommended that the DNA Advisory Board generate guidelines
for more stringent external case audits for use by ASCLD/LAB,
or another relevant accrediting body, as part of the accreditation
process. The external case audits should be conducted regularly
and serve as a measure of how well accreditation and its associated
requirements are working in a quality assurance context.
3. In the extreme, blind proficiency testing is possible, but
fraught with problems (including costs), and it is recommended
that a blind proficiency-testing program, be deferred for now
until it is more clear how well implementation of the first two
recommendations are serving the same purposes as blind proficiency
testing."
After a review of the NIJ
project, the Board issued the following resolution on blind external
proficiency testing on April 23, 1999:
"The DNA Advisory Board
accepts and adopts the finding of the NIJ Project that a blind
DNA proficiency testing program is not feasible at this time.
Although the DNA Advisory Board is supportive of the recommendations
proposed in the Final Report, the Board suggests that ASCLD/LAB
is in a better position to address the second recommendation."
In compliance with the DNA
Identification Act, the Board provided the Director of the
FBI with a recommendation for a system for grading proficiency-testing
performance to determine whether a laboratory is performing acceptably:
"All proficiency tests are graded as satisfactory or unsatisfactory.
A satisfactory grade is attained when there are no analytical
errors for the DNA profile typing data. Administrative errors
shall be documented and corrective actions taken to minimize
the error in the future."
According to the requirements
of the DNA Identification Act, the Board will terminate
on March 9, 2000, unless the Board's term is extended by the
Director of the FBI. The Board is currently evaluating and will
make recommendations concerning the certification of forensic
DNA personnel and statistical and population genetic issues related
to calculations involving uniqueness, sample mixtures, parentage,
and CODIS database searches.
Back to index
Forensic Science Laboratory Accreditation Criteria
and ISO 17025
J. Evans
National Association of Testing Authorities
Sidney, New South Wales, Australia
In Australia, all laboratories
accredited by the National Association of Testing Authorities
(NATA) including forensic science laboratories must comply with
the international laboratory accreditation standard ISO
Guide 25, soon to be renamed ISO 17025. ISO 17025 is a generic
standard for laboratories and is applicable to all facilities
performing tests and calibrations regardless of size or type
of service. Although it can stand alone as a standard describing
the elements of a laboratory's operation, it makes specific provision
for what it calls amplification or interpretation to ensure that
the requirements of the standard are applied in a consistent
manner.
The first ISO Guide 25 published
in 1978 and the second ISO Guide 25 published in 1982 covered
only technical requirements, that is the technical aspects of
a laboratory's operation. The third and current ISO Guide 25
was published in 1990. At that time, there was a recognition
of the need for laboratories to have in place a quality management
system. This was in line with the trend whereby organizations
of all types were establishing quality management systems that
complied with the ISO standard for quality management systems,
ISO 9000. The 1990 ISO Guide 25 combines quality management system
elements with technical elements. As a result, the 1990 ISO Guide
25 includes a considerably greater number of elements than the
two previous versions.
Since 1990, the introduction
of quality management systems by laboratories has increased greatly.
In addition, the number of organizations that have sought certification
to the quality management system standards ISO 9000 has also
increased dramatically. With this focus internationally on the
implementation of quality management systems, either at the broad
organizational level or at the laboratory level, ISO felt that
its Guide 25 needed revision to strengthen the quality management
system elements and to harmonize or align them more closely with
ISO 9000. This without diminishing in any way the technical requirements
of the Guide. The publication of ISO 17025 is expected toward
the end of 1999.
ISO 17025 will be divided
into two sections: one covering quality management system elements
and one covering technical requirements. The general format of
the latest draft of ISO 17025 is as follows:
- Management Requirements
- Organization and management
- Quality system
- Document control
- General
- Document approval and issue
- Document changes
- Request, tender, and contract
review
- Subcontracting of tests
and calibrations
- Purchasing services and
supplies
- Service to client
- Complaints
- Control of nonconforming
work
- Corrective action
- General
- Cause analysis
- Monitoring of corrective
action
- Special audits
- Preventive action
- Records
- General
- Technical records
- Internal audits
- Management reviews
- Technical Requirements
- General
- Personnel
- Accommodation and environmental
conditions
- Testing and calibration
methods including sampling
- Equipment
- General
- Selection of methods
- Laboratory-developed methods
- Nonstandardized methods
- Validation of methods
- Best measurement capability
- Estimation of uncertainty
of measurement
- Control of data
- Equipment
- Measurement traceability
- General
- Specific requirements
- Calibration
- Testing
- Reference standards and
materials
- Intermediate checks
- Transport and storage
- Sampling
- Handling and transporting
items
- Assuring the quality of
results
- Reporting results
- General
- Test reports and calibration
certificates
- Opinions and interpretations
- Results obtained from subcontractors
- Electronic transmission
of results
- Format of reports and certificates
- Amendments to reports and
certificates
The accreditation criteria
for NATA's forensic science laboratory accreditation program
are based on ISO 17025. For the additional interpretative criteria,
NATA has drawn on a number of sources, including the forensic
science laboratory accreditation programs of ASCLD/LAB, the United
Kingdom Accreditation Service, and the Standards Council of Canada,
in addition to developing our own.
In an effort to make the
accreditation criteria as user-friendly as possible to as many
laboratories as possible, the document has been structured in
a way that would be logical to forensic science practitioners,
and it includes a considerable amount of commentary or explanation.
The format of the NATA accreditation
criteria is as follows:
- Organization and Management
- General
- Objectives
- Administrative practices
- Organizational structure
- Delegation of authority
- Supervision
- General
- Multisite laboratories
- Communication
- Quality Management System
- General
- Documentation
- Quality manual
- Document and information
control
- Records
- General
- Case records
- Computers
- Internal audits
- Management review
- Complaints
- Corrective action
- Control of nonconforming
work
- Corrective action
- Preventive action
- Personnel
- Qualifications
- Management
- Analysts/Examiners
- Controlled substances
- Toxicology
- Trace evidence
- Serology
- DNA
- Firearms/toolmarks
- Questioned documents
- Latent prints
- Crime scene investigation
- Technical support (nontestifying
staff)
- Training and Development
- Evidence Management
- Evidence control
- Evidence retention and disposal
- Methods and Procedures
- General
- Documentation
- Method validation
- Reference materials
- Standards and reagents
- Subcontracting of technical
work
- Quality Assurance
- Quality control
- Proficiency testing
- Case record review
- General
- Technical review
- Administrative review
- Court testimony monitoring
- Equipment
- Reports
- Procurement of Services
and Supplies
- Accommodation and Safety
- Security
- Environmental conditions
- Space
- Design
- Health and safety
Back to index
FORENSIC SCIENCE COMMUNICATIONS JULY 1999 VOLUME
1 NUMBER 2 |
