Forensic Science Communications - July 2004
July 2004 - Volume 6 - Number 3
Standards and Guidelines
Revised Validation Guidelines
The validation section of the Guidelines for a Quality Assurance Program for DNA Analysis by the Technical Working Group on DNA Analysis Methods (Crime Laboratory Digest 1995:22(2):21-43) has been revised due to increased laboratory experience, the advent of new technologies, and the issuance of the Quality Assurance Standards for Forensic DNA Testing Laboratories by the Director of the FBI (Forensic Science Communications available: www.fbi.gov/hq/lab/fsc/backissu/july2000/codis2a.htm).
This document provides validation guidelines and definitions approved by SWGDAM July 10, 2003.
1. General Considerations for Validation of the DNA Analysis Procedure
1.1 Validation is the process by which the scientific community acquires the necessary information to
(a) Assess the ability of a procedure to obtain reliable results.
(b) Determine the conditions under which such results can be obtained.
(c) Define the limitations of the procedure.
The validation process identifies aspects of a procedure that are critical and must be carefully controlled and monitored.
1.2 There are two types of validation required to implement or modify technologies for forensic DNA analysis—developmental and internal. The application of existing technology to the analysis of forensic samples does not necessarily create a new technology or methodology. Developmental validation studies in other fields sufficiently address forensic applications.
1.2.1 Developmental validation is the demonstration of the accuracy, precision, and reproducibility of a procedure by the manufacturer, technical organization, academic institution, government laboratory, or other party. Developmental validation must precede the use of a novel methodology for forensic DNA analysis.
220.127.116.11 Peer-reviewed publication of the underlying scientific principle(s) of a technology is required.
18.104.22.168 Peer-reviewed publication of the results of developmental validation studies is encouraged. However, technologies or procedures may be implemented without peer-reviewed publication if the results of developmental studies have been disseminated to the scientific community for review and evaluation through multiple ways, such as presentation at a scientific meeting or publication in a technical manual.
22.214.171.124 Internal validation studies must be sufficiently documented and summarized.
126.96.36.199 Internal validation should lead to the establishment of documented quality assurance parameters and interpretation guidelines.
188.8.131.52 Satellite laboratories must perform an internal validation independent of the main laboratory. Performance-based tests must be completed and documented for each laboratory location, whereas basic validation data may be shared by all locations in a laboratory system.
184.108.40.206 A complete change of detection platform or commercial kit requires an internal validation.
2. Developmental Validation: The developmental validation process may include the studies detailed below. Some studies may not be necessary for a particular method.
2.1 Characterization of genetic markers: The basic characteristics (described below) of a genetic marker must be determined and documented.
2.1.1 Inheritance: The mode of inheritance of DNA markers demonstrated through family studies.
2.1.2 Mapping: The chromosomal location of the genetic marker (submitted to or recorded with the Nomenclature Committee of the Human Genome Organization).
2.1.3 Detection: Technological basis for identifying the genetic marker.
2.1.4 Polymorphism: Type of variation analyzed.
2.2 Species specificity: For techniques designed to type human DNA, the potential to detect DNA from forensically relevant nonhuman species should be evaluated. For techniques in which a species other than human is targeted for DNA analysis, the ability to detect DNA profiles from nontargeted species should be determined. The presence of an amplification product in the nontargeted species does not necessarily invalidate the use of the assay.
2.3 Sensitivity studies: When appropriate, the range of DNA quantities able to produce reliable typing results should be determined.
2.4 Stability studies: The ability to obtain results from DNA recovered from biological samples deposited on various substrates and subjected to various environmental and chemical insults has been extensively documented. In most instances, assessment of the effects of these factors on new forensic DNA procedures is not required. However, if substrates and/or environmental and/or chemical insults could potentially affect the analytical process, then the process should be evaluated using known samples to determine the effects of such factors.
2.5 Reproducibility: The technique should be evaluated in the laboratory and among different laboratories to ensure the consistency of results. Specimens obtained from donors of known types should be evaluated.
2.6 Case-type samples: The ability to obtain reliable results should be evaluated using samples that are representative of those typically encountered by the testing laboratory. When possible, consistency of typing results should be demonstrated by comparing results from the previous procedures to those obtained using the new procedure.
2.7 Population studies: The distribution of genetic markers in populations should be determined in relevant population groups. When appropriate, databases should be tested for independence expectations.
2.8 Mixture studies: The ability to obtain reliable results from mixed source samples should be determined.
2.9 Precision and accuracy: The extent to which a given set of measurements of the same sample agree with their mean and the extent to which these measurements match the actual values being measured should be determined.
2.10 PCR-based procedures: Publication of the sequence of individual primers is not required in order to appropriately demonstrate the accuracy, precision, reproducibility, and limitations of PCR-based technologies.
2.10.1 The reaction conditions needed to provide the required degree of specificity and robustness must be determined. These include thermocycling parameters, the concentration of primers, magnesium chloride, DNA polymerase, and other critical reagents.
2.10.2 The potential for differential amplification among loci, preferential amplification of alleles in a locus, and stochastic amplification must be assessed.
2.10.3 When more than one locus is coamplified, the effects of coamplification must be assessed (e.g., presence of artifacts).
2.10.4 Positive and negative controls must be validated for use.
2.10.5 Detection of PCR product
220.127.116.11 Characterization without hybridization
18.104.22.168.1 When PCR product is characterized directly, appropriate measurement standards (qualitative and/or quantitative) for characterizing the alleles or resulting DNA product must be established.
22.214.171.124.2 When PCR product is characterized by DNA sequencing, appropriate standards for characterizing the sequence data must be established.
126.96.36.199 Characterization with hybridization
188.8.131.52.1 Hybridization and wash conditions necessary to provide the required degree of specificity must be determined.
184.108.40.206.2 For assays in which the probe is bound to the matrix, a mechanism must be employed to demonstrate whether adequate amplified DNA is present in the sample (e.g., a probe that reacts with an amplified allele(s) or a product yield gel).
3. Internal Validation: The internal validation process should include the studies detailed below encompassing a total of at least 50 samples. Some studies may not be necessary due to the method itself.
3.1 Known and nonprobative evidence samples: The method must be evaluated and tested using known samples and, when possible, authentic case samples; otherwise, simulated case samples should be used. DNA profiles obtained from questioned items should be compared to those from reference samples. When previous typing results are available, consistency as to the inclusion or exclusion of suspects or victims within the limits of the respective assays should be assessed.
3.2 Reproducibility and precision: The laboratory must document the reproducibility and precision of the procedure using an appropriate control(s).
3.3 Match criteria: For procedures that entail separation of DNA molecules based on size, precision of sizing must be determined by repetitive analyses of appropriate samples to establish criteria for matching or allele designation.
3.4 Sensitivity and stochastic studies: The laboratory must conduct studies that ensure the reliability and integrity of results. For PCR-based assays, studies must address stochastic effects and sensitivity levels.
3.5 Mixture studies: When appropriate, forensic casework laboratories must define and mimic the range of detectable mixture ratios, including detection of major and minor components. Studies should be conducted using samples that mimic those typically encountered in casework (e.g., postcoital vaginal swabs).
3.6 Contamination: The laboratory must demonstrate that its procedures minimize contamination that would compromise the integrity of the results. A laboratory should employ appropriate controls and implement quality practices to assess contamination and demonstrate that its procedure minimizes contamination.
3.7 Qualifying test: The method must be tested using a qualifying test. This may be accomplished through the use of proficiency test samples or types of samples that mimic those that the laboratory routinely analyzes. This qualifying test may be administered internally, externally, or collaboratively.
4. Material Modification: A material modification is a substantial and/or consequential alteration of a physical or analytical component in an integrated procedure. The modified procedure must be validated as concomitant with the nature of the alteration.
4.1 Commercial manufacturers should notify users of any material modifications made to products.
4.2 Modified procedures must be performance evaluated by comparison with the original procedure using similar DNA samples.
5. Performance Check of Established Procedures: A performance check is an evaluation of a validated procedure existing in the laboratory system to ensure that it conforms to specifications.
If a laboratory changes its physical location or its infrastructure has been substantially changed, a performance check regarding reproducibility and sensitivity must be completed.
5.1 Each new instrument or software change (including upgrades) requires a performance check.
Accuracy: The extent to which a given measurement matches the actual value being measured.
Analytical procedure: An orderly step-by-step procedure designed to ensure operational uniformity and minimize analytical drift.
Contamination: The unintentional introduction of exogenous DNA into a DNA sample or PCR reaction prior to amplification.
DNA type: The genetic constitution of an individual at defined locations (also known as loci) in the DNA. A DNA type derived from nuclear DNA typically consists of one or two alleles at several loci (e.g., short tandem repeat loci). The DNA type derived from mitochondrial DNA is described in relation to the revised Cambridge Reference Sequence (Nature Genetics 1999, 23, 147).
Forensic DNA analysis: The process of characterizing DNA obtained from human biological samples (e.g., obtained from evidentiary material from crime scenes, suspects, victims, and convicted offenders) for application to questions of criminal law. The process results in the determination of a DNA type at defined locations in the DNA.
Hybridization: The process of complementary base pairing between two single strands of DNA and/or RNA.
Material modification: Alteration of an existing analytical procedure that may have a consequential effect(s) on analytical results.
Method: A system of analysis executed using an ordered series of steps.
Peer review: Review of data, documentation, and reports by a second qualified person to check for consistency, accuracy, and completeness. Both people (reporting analyst and peer reviewer) must agree on the interpretation of the data and the conclusions derived from the data.
Performance check: A quality assurance measure to assess the functionality of laboratory instruments and equipment that affect the accuracy and/or validity of forensic casework or convicted offender examinations.
Polymerase chain reaction: An enzymatic process by which a specific region of DNA is replicated, or amplified, during repetitive cycles to yield many copies of a particular sequence. A PCR cycle consists of the following three steps.
- Denaturation or conversion of the double-stranded template DNA into its constituent single strands.
- Annealing of primers to complementary sequences in the DNA template.
- Extension of the bound primers by a DNA polymerase.
Polymorphism (genetic): The occurrence in a population of two or more alleles at a genetic locus, when the frequency of the most common allele is less than 99 percent.