Standards and Guidelines - Forensic Science Communications - April 2005

April 2005 - Volume 7 - Number 2

Standards and Guidelines

Validation Guidelines for Laboratories Performing Forensic Analysis of Chemical Terrorism

Scientific Working Group on Forensic Analysis of Chemical Terrorism (SWGFACT)

Preface

The Scientific Working Group on Forensic Analysis of Chemical Terrorism has developed the following guidelines for laboratories engaged in the forensic analysis of chemical evidence associated with terrorism. This document provides a baseline framework and guidance for a validation program for forensic analytical procedures. Consideration may be given to alternative approaches of achieving the intent of these guidelines.

Introduction

The Scientific Working Group on Forensic Analysis of Chemical Terrorism’s mission is “to develop guidelines for the forensic identification, characterization, and attribution of evidence in planned, threatened, or actual acts of chemical terrorism.” The Validation Guidelines for Laboratories Performing Forensic Analysis of Chemical Terrorism may be used by laboratories to structure or enhance their validation of procedures used in the forensic analysis of chemical terrorism.

Analytical procedure validation is often a complex, iterative process that requires scientific judgment. The goal of these guidelines is to provide the laboratory with information on steps that are commonly used when validating an analytical procedure. This guidance is meant to complement the experience and professional judgment of the laboratory personnel.

Chemicals associated with terrorism may present acute hazards not normally encountered during routine operations by laboratory personnel. Appropriate caution and safety should be exercised when dealing with these types of hazardous materials.

Reference

Scientific Working Group on Forensic Analysis of Chemical Terrorism, Quality assurance guidelines for laboratories performing forensic analysis of chemical terrorism, Forensic Science Communications [Online] (2004).

1. Scope

These guidelines describe validation practices that a laboratory should follow to ensure that its analytical procedures produce data that are fit for the intended purpose. It is the responsibility of each laboratory to select the appropriate analytical procedure that will meet the intended data quality objectives. A laboratory, in the context of these guidelines, is defined as a facility in which analysis associated with chemical terrorism is performed. It is assumed that the laboratory is operating under the quality assurance system described in the Quality Assurance Guidelines for Laboratories Performing Forensic Analysis of Chemical Terrorism.

This guidance may be useful in the following situations:

When a standardized analytical procedure is applied in the laboratory for the first time.

When a standardized analytical procedure is modified.

When a published but nonstandardized analytical procedure is used in the laboratory.

When an analytical procedure is developed in-house for subsequent routine use.

When an analytical procedure is developed in-house as a solution to a unique analytical problem.

2. Definitions

As used in these guidelines, the following terms shall have the meanings specified. (See also Appendix A and Appendix B.)

Accuracy is the extent to which an analytical result approaches the true value. Accuracy includes a combination of random error (precision) and systematic error (bias).

Analyte is the chemical entity whose presence and/or concentration is being established.

Analytical procedure is an orderly step-by-step instruction designed to ensure operational uniformity and to minimize uncertainty.

Laboratory is a facility where chemical forensic testing is performed.

Limit of detection is the lowest concentration or smallest amount of analyte that can be statistically differentiated from the analyte-free sample matrix.

Limit of quantitation is the lowest concentration or smallest amount of analyte that can be measured at a specified accuracy.

Linearity is the extent to which an analytical procedure produces a signal directly proportional to the concentration or mass of the analyte of interest.

Matrix is the medium or material that may contain the target analyte(s).

Reference material is a material for which component identities, types, or values are certified by technically valid procedures and is accompanied by or traceable to a certificate or other documentation that is issued by a certifying body.

Sample is the subject of an analytical procedure. The sample consists of potential target analyte(s) that may exist in a matrix.

Selectivity is the extent to which an analytical procedure is free from interferences arising from nonanalytes, including matrix components.

Sensitivity is the level of instrumental signal obtained per unit amount of analyte. Sensitivity is not interchangeable with limit of detection.

Spike is a laboratory-prepared sample to which a known amount of the analyte of interest is added, either to a portion of the original sample or similar matrix.

Standard is a substance of known identity and purity and/or concentration.

Standardized analytical procedure is one that has been documented, validated, and approved by a recognized technical organization (e.g., AOAC International, ASTM International, U.S. Environmental Protection Agency, U.S. Pharmacopeial Convention Incorporated).

Validation is the process by which an analytical procedure is evaluated to determine its fitness for its intended use.

Working range is the concentration range over which the analytical procedure has been validated.

3. Validation Process

3.1. The laboratory should establish and maintain a documented validation process for the forensic analytical procedures performed by the laboratory.

3.2. The validation process should consist of the following:

Define the scope of the analytical procedure.

Identify the analytical procedure performance characteristic(s) that must be validated.

Select experiments to determine the required performance characteristic(s).

Conduct experiments to determine required performance characteristics.
Document, review, and maintain analytical procedure validation results.

4. Define the Scope of the Analytical Procedure

4.1. The laboratory should develop a clear and unambiguous statement defining the scope, which includes the matrix, target analyte(s), analytical technique, and intended purpose of the analytical procedure.

4.2. The purpose of an analytical procedure may include the following:

Identification of unknown component(s).

Establishment of the presence or absence of specified analyte(s) and/or classes.

Quantitation of specified analyte(s).

Determination of a physical and/or chemical property (e.g., mass, color, viscosity, flash point, particle morphology, crystalline structure).

5. Identify the Analytical Procedure Performance Characteristic(s) That Should be Validated

The performance characteristics that should be validated will vary depending on the purpose of the procedure and may include accuracy (precision and bias), limit of detection, limit of quantitation, linearity, working range, and selectivity. Selecting performance characteristics requires professional judgment. The following performance characteristics may be relevant to the intended purposes stated above:

5.1. Identification of unknown component(s).

Due to the nature of unknown component identification, the validation may be conducted either at the time of analysis or immediately following. The performance characteristic that should be validated in this case is selectivity.

5.2. Establishment of presence and/or absence of specified analyte(s) or classes including the following:

Selectivity.
Limit of detection.

5.3. Quantification of specified analyte(s) including the following:

Selectivity.

Linearity (or other calibration model).

Working range.

Limit of detection.

Limit of quantitation.
Accuracy.

5.4. Measurement of a physical property including the following:

Linearity (or other calibration model).

Working range.
Accuracy.

6. Select Experiments to Determine the Required Performance Characteristic(s)

The laboratory should evaluate the existing documentation relating to the analytical procedure to determine if it fulfills the performance characteristics relevant for the intended purpose described in Section 5. Existing validation data will vary based on the analytical procedure category, as described below, and may determine the need for additional experiments. In evaluating existing data, the laboratory should consider issues related to false positives and false negatives as described in Appendix B.

A standardized analytical procedure has been documented, validated, and endorsed by a recognized technical organization (e.g., AOAC International, ASTM International, U.S. Environmental Protection Agency, U.S. Pharmacopeial Convention Incorporated). In this case, the laboratory should demonstrate its capability to obtain similar performance characteristics.

A modified standardized analytical procedure is one that has been modified outside the specifications of the standardized procedure. In this case, the laboratory should verify that the modifications do not alter the performance characteristics such that the data are no longer fit for the intended purpose.

A nonstandardized analytical procedure has been developed externally but has not been previously endorsed by a recognized technical organization (e.g., an analytical procedure published in a technical journal). In this case, the laboratory should determine the performance characteristics applicable to the intended purpose and conduct any necessary validation experiments.
An in-house analytical procedure is developed in the laboratory for subsequent routine use or as a solution to a unique analytical problem. In this case, the laboratory should determine the performance characteristics applicable to the intended purpose and conduct any necessary validation experiments.

7. Conduct Experiments to Determine the Required Performance Characteristic(s)

The experiments required for validation depend on the analytical procedure scope, its intended use, the analytical procedure category, and the quality of the existing validation efforts. Validation requirements are often project-specific and may be adjusted depending on specific needs (e.g., safety considerations, limited quantity of available sample, differences in sample matrices, reference materials).

Laboratories should refer to Appendix A when conducting validation experiments for the various performance characteristic(s) and to Appendix B for considerations relating to false positives and false negatives. In addition, laboratories should refer to the Quality Assurance Guidelines for Laboratories Performing Forensic Analysis of Chemical Terrorism prior to conducting validation experiments.

8. Document, Review, and Maintain Analytical Procedure Validation Results

All documentation related to validation studies should be logically and consistently maintained in accordance with the Quality Assurance Guidelines for Laboratories Performing Forensic Analysis of Chemical Terrorism. This documentation includes references, analytical procedures, and experimental data. All known limitations of the procedure should be included in the documentation. Appendix C may be used as a template for summarizing analytical procedure validation data.

The validation documentation and data should be reviewed by a technically qualified person to ensure that the validation approach produces performance characteristics and that results are fit for the intended use of the analytical procedure. Evidence of the technical review should be maintained for future reference.

Appendix A: Analytical Performance Characteristics

Accuracy

Accuracy is the extent to which an analytical result approaches the true value. The accuracy of an analytical measurement is related to the random error (precision) and the systematic error (bias).

Precision determination is made by repeating a measurement over a specified time frame appropriate for the intended analytical procedure use. The measure of precision will depend on the range of conditions (e.g., analyte concentrations, matrices, instrumental parameters) over which the analytical procedure is applied. The measurements should be made using the entire analytical procedure including all preparation and analysis steps.

Because the true value need not be known, a wide variety of materials may be used to assess precision, including reference materials, in-house quality control materials, and the actual samples of interest. Precision is typically expressed as the percent relative standard deviation (% RSD):

% RSD = standard deviation of measurements * 100% mean of measurements

Bias in an analytical procedure is determined by comparing the measurement result with the true value. Bias can be estimated by measuring materials of known composition, such as reference materials. Matrix matched reference materials are considered the preferred materials for estimating bias. When a suitable reference material is not available, bias may be estimated by the analysis of spiked samples. The behavior of the added analyte may differ from that of the native analyte, but spiking attempts to achieve the goal of matrix matching. Spike recovery is calculated as follows:

% spike recovery = (measured concentration spiked sample - measured concentration unspiked sample) * 100% concentration of spike contribution

Bias can also be estimated by comparing results obtained for the same samples using another analytical procedure with a known bias (i.e., a reference method).

Limit of Detection

The limit of detection is the lowest concentration or smallest amount of analyte that can be statistically differentiated from the analyte-free sample matrix. The limit of detection depends not only on the sensitivity but also on the instrumental noise and/or blank variability.

The instrumental limit of detection is a measure of instrument performance and is not sample matrix specific. It is a measure of either the instrumental signal-to-noise level or the variability of a standard blank. Of greater importance is the analytical procedure limit of detection. It incorporates not only the instrumental sensitivity and noise but also the variability induced by components of the sample matrix.

There are many approaches used to calculate the limit of detection; therefore, the laboratory should define its approach for determining a limit of detection.

The limit of detection may or may not be relevant in a validation study, depending on the concentration range and the intended purpose of the analytical procedure. For example, for analytical procedures when the analyte measured is always in the calibration range of the assay and well above the true limit of detection, it may be sufficient to indicate that the detection limit is “less than” the value of the lowest nonzero calibration standard.

Limit of Quantitation

The limit of quantitation is the lowest concentration or smallest amount of analyte that can be measured at a specified accuracy. There are many approaches used to calculate the limit of quantitation; therefore, the laboratory should define its approach for determining a limit of quantitation.

The limit of quantitation may or may not be relevant in a validation study, depending on the concentration range and the intended purpose of the analytical procedure.

Linearity (or other calibration model)

Linearity is the extent to which an analytical procedure produces a signal directly proportional to the concentration or mass of the analyte of interest. Linearity (or other calibration model) is assessed by constructing a calibration curve (response versus analyte concentration) from known standards. Linear calibration models are frequently used, although various analytical procedures may yield acceptable nonlinear calibrations. The analyst can evaluate the linearity of the calibration curve by visual inspection or by using appropriate statistical methodology. The magnitude of the linear correlation coefficient, whereas sometimes used as a linearity measure, can be misleading. Depending on the number and spacing of calibration points, a visually nonlinear plot can lead to a correlation coefficient very close to one.

Working Range

Working range is the concentration or measurement range over which the analytical procedure has been validated. The concentration of the analyte of interest will have an effect on most analytical performance characteristics. Therefore, the analytical procedure should be validated for a working range consistent with its intended purpose.

The low end of the working range depends on the purpose of the analytical procedure. For example, if the purpose of the analysis at low concentrations is to simply indicate presence or absence of analyte, then the limit of detection may mark the low end of the working range. When accurate concentration values are needed, then the limit of quantitation may become the low end of the analytical procedure’s working range.

If the analyte response exceeds the working range, the working range should be reestablished. A more common approach is to dilute the sample into the working range.

Selectivity

Selectivity is the extent to which an analytical procedure is free from interferences arising from nonanalytes, including matrix components.

Although it is possible to establish that an interference exists, it is more difficult to state that no interferences exist. Matrix interferences are usually sample specific and should be addressed on a matrix-by-matrix basis. Many instrumental analytical procedures have specific approaches that can be used to detect (and possibly circumvent) lack of selectivity. Some examples are the use of an alternate column in a chromatography method or the use of alternate emission lines in emission spectroscopy.

Another approach to assessing selectivity is using an alternate analytical procedure for reanalysis of the samples. This assessment is the most convincing when an independent or orthogonal analytical technique is employed. Orthogonal techniques respond to distinct characteristics of a particular analyte. Infrared spectroscopy and mass spectrometry are orthogonal to each other, whereas infrared and Raman spectroscopies are not orthogonal to each other.

Appendix B: False Positives and False Negatives

Several quality control considerations are especially important when applying an existing method to a matrix or analyte for which the analytical approach has not been validated. They should also be incorporated in a regular quality control program.

The negative control (matrix blank) is a sample that closely matches the samples being analyzed with regard to matrix components and is collected to establish the background level (presence and/or absence), of the analyte(s) of interest. It incorporates all the reagents employed in treating the samples of interest and is subjected to all sample-processing operations. Its role is to verify that the normal sample matrix does not interfere with or affect the analytical signal. The negative control may be difficult to obtain because many matrices cannot be closely matched or guaranteed to be free from analytes.

The method blank is a quality control sample that incorporates all the reagents employed in treating the samples of interest and is subjected to all sample processing operations. A method blank serves to verify that an identified component does not originate in the reagents, by cross contamination, or from the analytical process.

A positive control is a quality control sample containing the target analyte(s) and is subjected to all sample-processing operations. The positive control may be a spiked matrix similar to the one being analyzed, or it may be a reference material. The positive control serves to demonstrate that the analyte of interest would have been detected, if present, at or above a particular concentration.

Carryover is the addition of analyte from a sample or standard to subsequent samples in a series of analyses. Carryover should be evaluated as a part of the validation effort. Carryover can often occur during instrumental analysis. The incorporation of appropriate blanks at key points in the analytical workflow (i.e., after analyzing the standards) could demonstrate the absence of analyte carryover.

Appendix C

Table 1: Summary Guidelines for Validating an Analytical Procedure

Procedure Purpose	Describe the purpose, which may include identifying unknown component(s); establishing presence and/or absence of specified analyte(s) and/or classes; quantifying specified analyte(s); and/or determining a physical property (e.g., mass, color, viscosity, flash point, particle morphology, crystalline structure).
Procedure Scope	Develop a clear and unambiguous statement defining the scope, which includes the matrix, target analyte(s), analytical technique, and intended purpose of the analytical procedure.
Selectivity	Describe the requirements of the analytical procedure with regard to selectivity. Describe experiments that should be performed to achieve the required selectivity. Consider the availability of reference materials, standards, matrix blanks.
Bias	Describe approaches used to assess bias, such as reference materials, alternate analytical procedures, or spike recovery.
Precision	Describe precision estimates, and state the range of conditions (e.g., analyte concentrations, matrices, instrumental parameters) over which the analytical procedure was validated.
Limit of Detection	Define the approach for estimating the limit of detection or provide a statement when this performance characteristic is not relevant to a validation.
Limit of Quantitation	Define the approach for determining the limit of quantitation.
Working Range	Describe the working range over which the analytical procedure was validated.
Calibration Model	Describe the type of calibration model acceptable to validate the analytical procedure. Assess linearity if applicable.
Critical Step	Provide any information about any steps that may be critical to the successful application of the analytical procedure.
Limitations	Identify all known limitations of the analytical procedure and its use. References list any previously documented analytical procedures referred to during the validation process. Include in validation documentation copies of analytical procedures that are not readily available.

Contributors

The following SGWFACT members contributed to this document:

Dean D. Fetterolf
Federal Bureau of Investigation
Laboratory
Quantico, Virginia

Armando Alcaraz
Lawrence Livermore National Laboratory
Forensic Science Center
Livermore, California

Shauna Darby
Battelle
Arlington, Virginia

Mary Drummond
Research, Development and Engineering Command
Forensic Analytical Center
Aberdeen Proving Ground, Maryland

Martin Harper
Centers for Disease Control and Prevention
National Institute for Occupational Safety and Health
Health Effects Laboratory Division
Morgantown, West Virginia

Charles Hooper
U.S. Environmental Protection Agency
Athens, Georgia

Eugene Kennedy
Centers for Disease Control and Prevention
National Institute for Occupational Safety and Health
Chemical Exposure and Monitoring Branch
Cincinnati, Ohio

Mathew Magnuson
U.S. Environmental Protection Agency
National Risk Management Research Laboratory
Cincinnati, Ohio

Madeline A. Montgomery
Federal Bureau of Investigation
Laboratory
Quantico, Virginia

Eric Nottingham
U.S. Environmental Protection Agency
National Enforcement Investigations Center
Denver, Colorado

James Peterson
Batelle
Aberdeen Proving Ground, Maryland

Charles F. Quenzer
Federal Bureau of Investigation
Laboratory
Quantico, Virginia

R. Duane Satzger
U.S. Food and Drug Administration
Forensic Chemistry Center
Cincinnati, Ohio

James Seidel
U.S. Environmental Protection Agency
Office of Criminal Enforcement
Forensics and Training
Denver, Colorado

Karen Wolnik
U.S. Food and Drug Administration
Forensic Chemistry Center
Cincinnati, Ohio

Sections