Home About Us Laboratory Services Forensic Science Communications Back Issues April 2001 Interactive CD-ROM Proficiency Testing and Training:...
Info
This is archived material from the Federal Bureau of Investigation (FBI) website. It may contain outdated information and links may no longer function.

Interactive CD-ROM Proficiency Testing and Training: Learning From Experience, by Ashley (Forensic Science Communications, April 2001)

Interactive CD-ROM Proficiency Testing and Training: Learning From Experience, by Ashley (Forensic Science Communications, April 2001)
fsc_logo_top.jpg
fsc_logo_left.jpg

April 2001 - Volume 3 - Number 2

Research and Technology

Interactive CD-ROM Proficiency Testing and
Training: Learning From Experience

Wayne Ashley
Senior Sergeant, Officer in Charge
Crime Scene Unit
Victoria Forensic Science Centre
Victoria Police
Victoria, Australia

Introduction | Proficiency Test | Proficiency Training
Future Directions
| References | Addendum A | Addendum B

Introduction

Proficiency testing in any discipline of forensic science is a management tool that monitors and measures the performance of an investigator or a facility and identifies key areas where improvement may be required. Proficiency testing can also measure quality systems and how the facility performs against other facilities in external proficiency-testing programs.

Crime scene examination is a subjective discipline that has been supported by scientific methods and quality assurance programs during the past 25 years. However, the interpretation of evidence at a crime scene is still primarily reliant on the observations of a trained investigator. Horswell (1995) recognized it in the following way:

Crime scene examination is pivotal to all forensic examination. Many criminal cases have demonstrated that the examination and analyses that follow any crime scene examination cannot be corrected in the laboratory if inadequate, incorrect or poorly performed procedures are adopted at the scene.

Crime scene examination is a forensic science activity that helps investigators identify, interpret, and recover physical evidence from a crime scene so that the evidence can be physically analyzed by the investigator or forwarded to the appropriate scientific discipline for further scientific analysis. The investigator is also responsible for the management of the crime scene, any attending scientific specialists, and the case management of evidence. In order to undertake this responsibility, the investigator must have the competence to record the crime scenes, as required, by notes, diagrams, photography, and video. The collection procedures must be of the highest level in order to provide integrity of exhibits and prevent any cross-contamination, particularly with trace evidence (Horswell and Edwards 1997).

The introduction of crime scene investigation training and external courses will give the investigator the requisite knowledge to add to his/her policing and investigative experience. Most training and skill development are competency-based. There has not been a way to monitor the performance of personnel or agencies in crime scene investigation until recently.

The Australian National Association of Testing Authorities, as a result of a recommendation from the Forensic Science Accreditation Advisory Committee, endorsed crime scene investigation as part of the criteria for accrediting forensic laboratories. Therefore, it was imperative that proficiency-testing programs be developed and delivered.

The Crime Scene Proficiency Advisory Committee was formed under the auspices of the National Institute of Forensic Science and is composed of five senior crime scene investigators from throughout Australia. The Crime Scene Proficiency Advisory Committee is responsible for developing all external proficiency tests relating to crime scene investigation. The committee will provide at least an annual proficiency test that will cover any or all of the following performance criteria:

  • Initially assessing the crime scene
  • Controlling the crime scene
  • Examining the crime scene
  • Interpreting evidence at the crime scene
  • Recording the crime scene and evidence
  • Collecting the evidence
  • Case management

The proficiency test scenarios developed must be representative of those encountered in crime scenes and reflect the jurisdictional-specific roles of crime scene investigators throughout Australia (Crime Scene Proficiency Committee 1999B).

Proficiency Test

Initial attempts to create suitable proficiency tests used a video format with written questions to be answered by participants. These tests were easy to prepare, but they restricted the crime scene investigator’s access to other areas of the scene. Assessment of competency from the written answers was also difficult. More recent attempts by Crime Scene Proficiency Advisory Committee to create plausible proficiency tests have used CD-ROM technology.

The concept of interactive CD-ROM technology is not new, and developing various crime scenes was not difficult. However, the need to depict reality, and the ability to move throughout a crime scene and process the evidence as one would at a crime scene proved to be a challenge. Upon completion, the interactive CD-ROM program After the Fact was distributed to 110 facilities throughout Australia in August 1999 (Crime Scene Proficiency Committee 1999A).

There are three aspects to the After the Fact crime scene proficiency test:

  • Crime scene virtual reality
  • Investigation tools
  • Written test

After the Fact allows an investigator to walk through a virtual crime scene and provides a realistic scene-processing function that includes the ability to take photographs, make notes, and collect and package evidence. Questions in accordance with the seven key criteria of crime scene investigation and relevant to that particular scene are presented to the investigator.

The initial screen in After the Fact, illustrated in Figure 1, shows various windows that deal with the three aspects of the proficiency-testing program. The crime scene virtual reality window shows the crime scene, and the investigator can move about the crime scene and zoom in or out on any area within the scene by using the keyboard or a mouse. The tools allow the investigator to process the scene as they would at a crime scene examination. Each tool allows provision in the larger split windows for descriptive text on any function (general notes, photographs, evidence collection, witness statements). The other split window provides instructions for using each tool. A third window allows the investigator to sequentially photograph collected evidence and review audio and video witness recordings. The test icon is separate from the virtual crime scene and should be attempted only when the investigator has finished processing the scene. The Quit icon allows exiting from the program.
The initial screen in the After the Fact crime scene investigation software.

Figure 1.
The initial screen in After the Fact. Click the image for a larger version.

The National Association of Testing Authorities mandates that an annual external proficiency test be taken. In order to provide an assessment process, which is acceptable to the National Association of Testing Authorities, the Crime Scene Proficiency Advisory Committee devised an achieved or not achieved grading system for the segments of the crime scene investigation process. For example, if an investigator collects evidence before making notes or taking photographs, he/she will receive a not achieved grade for that function. The exhibits identified need to be explained, recorded, and collected in the appropriate manner. An audit trail depicting the movement and actions of the investigator throughout the proficiency test provides the Crime Scene Proficiency Advisory Committee with an effective assessment tool. The ability to download the assessment to a disk allows the assessor to view the scene, understand how the scene was processed, and review the responses to relevant questions.

The Crime Scene Proficiency Advisory Committee will produce a report that is ratified by the Proficiency Review Committee. The Crime Scene Proficiency Advisory Committee report provides a reference for agencies to compare their results against other participating agencies and also provides feedback to management regarding the key areas of crime scene investigation that may require diagnostic or corrective action. This assists in raising the level of competency, skill, and knowledge so national standards are attained.

A sample of the manufacturer’s information for the first CD-ROM proficiency test is in Addendum A.

The After the Fact CD-ROM program provides the following advantages to the investigator and the Crime Scene Proficiency Advisory Committee:

  • Ability to navigate the crime scene and record and collect evidence as in an actual crime scene
  • Visual display of the scene and evidence
  • Reduction in paper because each test is recorded on a disk
  • An audit trail that allows the Crime Scene Proficiency Advisory Committee to monitor where an investigator went in the scene and the sequence used in processing the scene
  • Portability and mobility of the test program
  • Ability to reuse the program for training or internal testing
  • Ability to change scenarios without affecting the format or function of the program

A number of disadvantages were experienced in the first After the Fact CD-ROM proficiency test. Some of the following have already been corrected—others will require further education and training:

  • Computer hardware was insufficient to run the program.
  • Review programs were accessed with difficulty because of software problems.
  • Demonstration test to become familiar with operating the crime scene virtual reality window was inaccessible. The first test gave instructions only.
  • Some computers did not have CD-ROM drives or sound cards.
  • The program crashed on several computers, probably because of insufficient computer power.
  • Other computers were configured so that they were incapable of running the software program.
  • Several facilities had difficulty saving their tests to a disk.
  • The crime scene virtual reality window should be larger.

A number of agencies completed the proficiency test and saved the test on a disk for review purposes. An unedited, completed competency test is in Addendum B.

Proficiency Training

It was acknowledged by the Crime Scene Proficiency Advisory Committee and the National Institute of Forensic Science that this proficiency testing had potential as a training tool. Therefore, a separate training package was developed so that crime scene agencies in Australia and around the world had access to a system that provided self-assessment and formative learning in crime scene investigation.

The use of the program with the crime scene virtual reality and window displays was ideally suited to providing a learning tool for instruction in crime scene investigation to new trainees or as a refresher for investigators who require assessment and development.

The program allows the investigator to undertake training in the following skills:

  • Initially assessing the crime scene
  • Controlling the crime scene
  • Examining the crime scene
  • Interpreting evidence at the crime scene
  • Recording the crime scene and evidence
  • Collecting the evidence
  • Case management

The investigator or the supervisor tutoring an individual or a classroom can use any or all of the investigative criteria listed previously. It is recommended that the criteria be followed in sequential order. It is important that the investigator or supervisor read and understand the instructional information before proceeding with the training program.

The training program (Figure 2) provides instructional information on the following:

  • Crime scene virtual reality
  • Seven investigative categories
  • Principles, questions, and answers

The window in Figure 2 depicts the crime scene virtual reality in the proficiency test program. The investigator can navigate the crime scene. There are general principles for crime scene investigation, and the use of the split windows allows for explanation of the general principles or any of the principles relating to the seven key areas of investigation. The investigator can read these principles as they apply to each key area and then answer questions relating to the principles and the crime scene scenario. The investigator can type his/her answers in the window next to the questions (Figure 3). The investigator can then check the responses against the desired responses identified in the answers text box. An audit trail of this training program can show the instructor how the investigator is progressing.

Screen depicting the crime scene virtual reality of After the Fact, which can be used as a training program.

Figure 2.
Screen depicting the crime scene virtual reality of After the Fact, which can be used as a training program. Click image for a larger version.
Screen showing how an investigator can type answers in the After the Fact window next to the questions.

Figure 3.
Screen showing how an investigator can type answers in the After the Fact window next to the questions. Click for enlarged image.


This training program can be used for any type of crime scene investigation or for any discipline within forensic science. It is currently being evaluated for fire, explosion, and clandestine laboratory training.

The training program was successfully demonstrated at the 83rd International Educational Conference sponsored by the International Association of Identification in Little Rock, Arkansas, in 1998. The participants represented law enforcement agencies throughout the world, and a number were not directly involved with crime scene investigations. Each participant was able to navigate the program and provide creditable answers to the crime scene questions relating to the key categories.

The advantages of this proficiency training include the following:

  • Interactive training using part or all of the key investigative categories
  • Individual or classroom training
  • Familiarization of interactive proficiency programs
  • Multiple use and reuse of differing crime scene scenarios
  • Multiple use and reuse of other forensic disciplines
  • Distance learning program
  • Reference-based

The disadvantages of this proficiency training include the following:

  • Dedicated hardware and software to run the program

The interactive training program is ideal because it provides the background for crime scene investigation, the principles behind the philosophy of physical evidence, and the non-physical processing of a crime scene. Participation at mock scenes or minor crime scenes under supervision is also an essential aspect of the learning process in crime scene investigation.

The use of the training programs can have a marked effect on proficiency testing and can complement the testing programs. Figure 4 explains how such a system could work.

Diagram showing a complementary system of training and proficiency testing.

Figure 4.
Diagram showing a complementary system of training and proficiency testing. Click image for larger version.


Future Directions

Proficiency testing and training programs will continue to provide the required quality management processes that are necessary for investigators or agencies subscribing to accreditation programs such as the National Association of Testing Authorities or those who wish to improve their quality assurance through continuous improvement and monitoring of investigators’ competencies.

Advances in technology, with 360-degree digital recording systems and clearer image analysis, will continue to support the framework already established for interactive assessment and training programs. Continual support and feedback from participating agencies will also provide valuable information to the Crime Scene Proficiency Advisory Committee so that they strive to provide the best system of proficiency testing and training for crime scene investigators throughout Australia.

For additional information, contact the author at: w.ashley@police.vic.gov.au.


References

Crime Scene Proficiency Advisory Committee. Proficiency Test Number 1/99. Report to Facilities. Unpublished, 1999A.

Crime Scene Proficiency Advisory Committee. Standard Operating Procedures. Unpublished, 1999B.

Horswell, J. Education and training of police in the forensic sciences: An Australian perspective, Science and Justice (1995) 35:15-18.

Horswell, J. and Edwards, M. Development of quality systems accreditation for crime scene investigators in Australia, Science and Justice (1997) 37:3-8.