NIST names OSAC Resource Committees

As many regular readers will recall, NIST is trying to end the Wild Wild West that is the modern practice of Forensic Science. Just moments ago, they released their latest appointees:

An initiative to strengthen and bring uniformity to forensic science standards took another step forward today as the National Institute of Standards and Technology appointed 35 new members to the Organization for Scientific Area Committees (OSAC).

The new members, selected for their expertise in law, psychology and quality assurance, will serve on three advisory committees. These OSAC Resource Committees will play a critical support role by advising the Forensic Science Standards Board, the scientific area committees and subcommittees focused on specific forensic science disciplines within OSAC as they adopt, develop and review standards.

“As our science-focused committees and subcommittees work to support the development of forensic science standards and guidelines, we expect that there will be many questions related to law, work flow processes and quality control. These resource committees will help address those,” said John Paul Jones II, associate director for OSAC affairs.

The Human Factors Committee will provide guidance on how systems design influences human performance, on how to minimize cognitive and confirmation bias, and on how to mitigate errors in complex tasks.

The Legal Resource Committee will review and provide a legal perspective on proposed standards.

The Quality Infrastructure Committee will assemble and update a Forensic Science Code of Practice, provide guidance on quality issues, and provide impact statements that inform agency management about how specific standards may affect laboratory operations. It will also work with outside standards development organizations and accrediting bodies as needed.

The resource committee members were chosen from among 1,300 OSAC applicants. They include public defenders, law school professors, prosecutors, judges, standards development experts, laboratory managers and human factors experts.

A NIST-DOJ membership selection team is reviewing applications for the remaining OSAC positions and will announce the appointments as they are completed.

To see the membership of each resource committee, please go to


Human Factors Committee Members

  • Deborah A. Boehm-Davis, Ph.D., Dean of the College of Humanities and Social Sciences, George Mason University
  • Itiel Dror, Ph.D., Principal Researcher, Cognitive Consultants International
  • Cleotilde Gonzalez, Ph.D., Associate Research Professor of Decision Sciences, Carnegie Mellon University
  • Christian A. Meissner, Ph.D., Professor of Psychology, Iowa State University
  • Erin Morris, Ph.D., Behavioral Sciences Research Analyst, Los Angeles County Public Defender
  • Sunita Sah, Ph.D., Assistant Professor, Strategy, Economics, Ethics and Public Policy at Georgetown University McDonough School of Business
  • Scott Shappell, Ph.D., Human Factors and Systems Department Chair, Emory-Riddle Aeronautical University
  • Dan Simon, Professor of Law and Psychology, University of Southern California, Gould School of Law, and Department of Psychology
  • Brian C. Stanton, cognitive scientist, National Institute of Standards and Technology
  • William C. Thompson, Ph.D., Professor of Criminology, Law, and Society and Psychology and Social Behavior and Law, University of California Irvine

Legal Resource Committee Members

  • Jennifer Friedman, Deputy Public Defender, Los Angeles County
  • Christine Funk, General Counsel, Department of Forensic Sciences, Washington, D.C. (local government)
  • Lynn Robitaille Garcia, General Counsel, Texas Forensic Science Commission (state government)
  • Ted R. Hunt, Chief Trial Attorney and DNA Cold Case Project Director, Jackson County Prosecutor’s Office, Kansas City, MO
  • John Kacavas, United States Attorney, U.S. Department of Justice
  • David H. Kaye, Professor, Graduate Faculty, Forensic Science Program, Pennsylvania State University
  • David A. Moran, Professor of Law, University of Michigan Law School
  • Christopher J. Plourd, Superior Court Judge, State of California
  • Ronald S. Reinstein, Judge and Judicial Consultant, Arizona Supreme Court
  • Barry Scheck, Professor, Cardozo School of Law, Yeshiva University; Co-Director, Innocence Project; Commissioner, NY Commission on Forensic Science; Neufeld, Scheck, & Brustin, LLC

Quality Infrastructure Committee Members

  • Karin Athanas, Program Manager, American Association For Laboratory Accreditation
  • Sally S. Aiken, Medical Examiner, Spokane County, Washington
  • Barbara E. Andree, Forensic Chemist, Bureau of Alcohol, Tobacco, Firearms and Explosives
  • Jason Bond, Quality Assurance Coordinator, Indiana State Police Laboratory Division
  • Pamela L. Bordner, Sr. Accreditation Program Manager, ASCLD/LAB
  • Kris Cano, Forensic Laboratory Manager, Scottsdale Police Department Crime Laboratory
  • Deborah Friedman, Criminalist III, Broward Sheriff’s Office Crime Laboratory
  • Matthew Gamette, Laboratory Improvement and Quality Manager, Idaho State Police Forensic Services
  • Keith Greenaway, Vice President, ANSI-ASQ National Accreditation Board
  • Arlene Hall, Commander, Illinois State Police, Division of Forensic Services
  • Bruce Houlihan, Director, Orange County Crime Laboratory/Orange County Sheriff-Coroner
  • Alice R. Isenberg, Ph.D., Section Chief, FBI Laboratory
  • Timothy Kupferschmid, Laboratory Director, New York City Office of Chief Medical Examiner
  • Karen Reczek, Senior Standards Information Specialist, NIST Standards Coordination Office
  • Frances E. Schrotter, Sr. Vice President and Chief Operation Officer, American National Standards Institute

The problems with the modern practice of forensic firearm and toolmark analysis

The field is technically called a forensic firearm and toolmark analysis whose experts use an optical comparison microscope to examine the exemplar with the unknown recovered from the crime scene by a comparison criteria that is referred to as  “sufficiently consecutive matching striae” in an attempt to determine unique source.


Which on is it from? Can science really tell?

Which one is it from? Can science really tell?

The modern practice of forensic firearm and toolmark analysis leaves a lot to be desired. A lot. All pattern recognition endeavors do as well (e.g., latent fingerprint, handwriting, hair comparison, odontology-bite mark, lead bullet). It was subject to a few blog posts here not too long ago


The notion that with no validated and standardized criteria that have been statistically proven to provide for uniqueness that an examiner who has no formal tribology background can bridge the large gap between the millions of firearms in that class down to not simply a subclass and not merely to a batch in a given day, but rather to a specific firearm that is exclusive of all other firearms on the planet earth is pure apocrypha. Fantasy really.


First we have to understand their historical method and their historical training:

Here is how they are typically trained to look at the casings (note that it is in 2D)


Basic nomenclature used by them:

  • Striations: Lines or scratches left by the irregularities on the end of the tool as it scrapes away metal.
  • Stamped or Impression markings: Toolmarks created by the striking of a cartridge primer by the firing pin, or the head of a cartridge case by the ejector.
  • Stamped Striation Marks: Impressions left by the firing pin, ejector, and breechblock, if these parts have been manufactured by filing, or turned on a lathe.


In very brief this is what they are trained to do:

They test fire several rounds of ammunition of the same brand as crime scene ammunition through the suspect firearm. This makes the exemplars. They examine and compare test firings first. If they “match” perfectly, they examine the evidence recovered from the crime scene.

They begin their examination by first aligning the ejector and extractor marks properly. Then examine the firing pin impressions, noting the shape, position, depth and width. Then look for irregularities such as tips or depressions within the firing pin crater as a whole. Finally, look for identifiable machine toolmarks in the firing pin crater. Then examine the breech face marks, making notations regarding the location, length, width and contour of toolmark impressions. Extractor and ejector marks can also be examined and compared in a similar manner.


Here are straie in case you have never seen them:



There are many issues with this method of analysis and in particular its interpretation:

1. This is not the 19th century. Firearms are now mass-manufactured, thus eliminating much of the “uniqueness” of a gun that came from being built by hand back then. There is very little hand filing if ever these days. The firearms are made using cutting tools made of High Speed Steel (HSS) (e.g., T-1, M-2, M-7, and M-42) or Carbide (made of Tungsten Carbide based alloys) such as the below:


Modern cutting tools do not wear anywhere near as rapidly as they once did. They are very rugged as they are constructed or coated with cobalt, or Titanium Nitride (TiN) which is added to HSS tools to increase hardness. Titanium Carbon-Nitride (TiCN) is added to HSS or carbide tools to increase hardness and prevent metal from adhering to the tool. Titanium Aluminum Nitride (TiAlN) increases tool life during high temperature cutting. Polycrystalline Cubic Boron Nitride (PCBN). So, if the tools that make the toolmarks in the firearm last longer and fail to deform more readily, by necessary implication the difference from firearm to firearm is less distinctive.

This all involves is the beautiful science known as tribology. In tribology there is an entire fascinating filed that involves the geometry of chip formation and Built-up edge (BUE formation) and the shear fracture left behind from the use of a tool such as this:


2. The core of the issue of real world identification is that as you fire a firearm each time it will “wear” down the barrel. Metal on metal under gas pressure will do that. This concept is well known in the true forensic science community of tribology where the literature is overwhelming that an identification of source is simply not possible if the firearm is within its first 300 or so discharges through the barrel. How does the government know that a given firearm is inside or outside of its first three hundred discharges? They don’t, they just assume. Or they don’t know this key fact.

3. There is also the concept of how precisely and under what conditions there can be toolmark transference. Toolmarks are created by metal cutting tools on the machined surfaces of firearms. These toolmarks are then transferred onto the cartridge case surfaces by the expansion of the cartridge brass in the barrel chamber, and breech face of the pistol slide. In the case of bullets, toolmarks are transferred onto the bullet surfaces by the obturation of the bullet into the rifled barrel lands and grooves. The modulus of elasticity of typical cartridge brass is 16×10^6 PSI, and the modulus of elasticity for typical gun steel is 29×10^6 PSI. Obviously the lower modulus of elasticity of brass causes the brass to expand under high pressure into the barrel chamber, and pistol breech face. Modulus of elasticity is defined as the ratio of the applied stress or force to the change in shape of an elastic body. NOT ALL TOOLMARKS TRANSFER. There is also the issue of frictional forces between the moving parts that can leave marks as well.

4. The interpretation is not standardized and non-validated. The simple point in fact is that despite what we in the lay world believe there is infrequently any criteria (e.g., 12 points of comparison) to make this all important call from class to uniqueness.

  • Class Characteristics: Marks that all firearms of a given make and model will leave. These are coarser toolmark impressions.
  • Subclass Characteristics: Fall under this category, are present in only some toolmarks. They are created during batch lot manufacturing of cutting tools, and are present on only those tools from a specific batch lot. These are also coarser toolmark impressions.
  • Individual Characteristics: Marks specific to only one firearm. These are finer toolmark impressions.
  • Accidental Characteristics: Non-reproducible toolmarks that may appear on one spent cartridge case, but not on any other spent cartridge case.

Again, no universal standards exist delineating these characteristics. If there are any, they are not statistically sound in terms of their validity.

Further, traditionally, breech face marks and firing pin impressions on a cartridge case head (and land and groove impressions on a bullet) are considered individual marks. But, not all breech face marks or land and groove impressions are in fact unique to that gun. Studies show that up to 20% matching striations may be all unique to that gun, but all common to several guns of the same make and model, or a combination of both. Again, there are no standards to determine which breech face marks (or land and groove impressions) are unique to that gun. Firearms examiners routinely just assume that any matching breech face marks identify the gun uniquely when looking at them in 2D.

5. The real problem for this science lies in the cognitive bias inherent in the examination itself. You, the examiner, fire a bullet from a known firearm. You retrieve it. You examine the seized evidence round that comes from a police investigation. Boy. Do you think that you are primed for cognitive bias?

Here is a case study of where the prosecution through PSP first examined a recovered firearm and called it a “match” to seized evidence. They had no photomicrographs, just an opinion by a high school graduate. It was reduced to this report. Note that it is specific as to source.

10 When challenged, they then went and took the unusual step of making photomicrographs off of their comparison microscope that resulted in this. Note it is in 2D (Of course, it was still considered a “match”) When pressed with precisely how they came up with this theory of uniqueness, it all came down to the most horrible phrase ever uttered in a courtroom “based upon my training, knowledge and experience.”

11 The top is the recovered evidence and the bottom is the exemplar from the recovered gun. Note they are not even at the same magnification or in the same orientation.

Instead, we used a Scanning Electron Microscope (the below results are from this same case and are of the same evidence and exemplar where PSP said that it was a “100% match”) was revealed in 3D by SEM to be far from it as seen below when examined in 3D and topographically examined when just looking at the firing pin position and characteristics in 3D alone. (Luckily for us in this case, this examiner had previously testified in another case that was provided to me by a PACDL member that if the firing pin position and characteristics were not a “match” then it was a “positive exclusion” of the firearm.)

20 21 22

6. All of this is why there is a big push for funding the studies to consider the issue of at what point do the similarities become enough to distinguish and form valid opinions of specificity. When studies involving white light inferometers were used to get a 3D image (instead of the 2D image that the comparison microscope uses), the revealing part was “that only 21% to 38% of the striae on pairs of bullets fired from the same” gun “matched.” Wow!


Here is a data sheet from one result:

31 Instead, what do we do? Often non-credentialed lay witnesses who were former traffic cops who are trained by others similarly lacking in an engineering or tribology background using a quasi-apprenticeship method, make the call of a “match.”  The term of art (or the call meaning the use of the word “match”) used by these practitioners is arrived at by these untrained microscopist looking with what amounts to their “highly trained set of eyeballs” at a three-dimensional object under a insufficiently magnified stereo comparison microscope which is a two dimensional representation to arrive at the conclusion that there is supposedly “sufficient consecutively matching striae” to warrant a conclusion that the unknown and the known are in fact a match.


Delaware State Police (“DSP”) and the Delaware Department of Justice (“DDOJ”) initiated an investigation of the Office of the Chief Medical Examiner (“OCME”) Controlled Substances Unit (“OCME-CSU” or “CSU”).

According to this news story, there were lots of problems found.

New details have emerged in an evidence scandal that occurred within in the Delaware Medical Examiner’s office.

A new report from Attorney General Beau Biden’s office revealed numerous “systemic operational failings” within the Chief Medical Examiner’s Controlled Substance Unit. These failings allegedly led to missing or altered drug evidence in 46 cases.

An audit discovered at least 51 pieces of potentially compromised evidence. The missing evidence includes marijuana, Oxycontin, heroin and cocaine.

The preliminary investigation, conducted by the Department of Justice and Delaware State Police, demonstrated the absence of management, oversight and security within the lab. Detailing lax security procedures, employees were described in the report as propping the drug vault door open and turning off the security alarm system.

“Employees recall having observed the door to the drug vault propped open numerous times over the years,” stated the report. “When the DSP secured the drug vault on February 20, 2014, a well-worn, wooden chock was observed in the area adjacent to the door. Based on witness interviews, investigators believe this was used to hold the door open.”

At times, drug evidence was not handled, stored or tested according to protocol, according to the investigation. Records were mismanaged and evidence was removed without being properly logged out. Additionally, the report stated that some employees lacked the training or experience needed to perform some of the tasks to which they were assigned.

The investigation has lead to the arrest of two lab employees, Forensic Investigator James Woodson and Laboratory Manager Farnam Daneshgar, along with the suspension of Chief Medical Examiner Dr. Richard Callery.

Woodson was indicted on one count each of trafficking cocaine, theft of a controlled substance, official misconduct, and tampering with evidence.

Daneshgar was indicted on two counts of falsifying business records. According to the report, witnesses accused Daneshgar of “dry labbing,” which is the “practice of declaring a result without performing the analytical testing to produce the result.”

Overhaul Planned

The report was released as Delaware lawmakers prepare to vote on a bill that would abolish the chief medical examiner’s office and create the Division of Forensic Science. A director would oversee the division, which would be housed under the Department of Safety and Homeland Security rather than the Department of Health and Social Services.

The division would be responsible for overseeing the chief medical examiner’s office and would be responsible for working with the courts and law enforcement, investigating deaths, participating on the Criminal Justice Council and providing fatal incident reviews to the Domestic Violence Coordinating Council.

“Delaware must have its own independent, state-of-the-art crime laboratory,” said Biden in a statement. “A new crime lab is the right thing for Delaware’s criminal justice system and the right thing for taxpayers.”

The legislation is being considered in the state Senate.



In case you have been living under a rock, you have heard of the NIST OSAC movement to try to reform and standardize the practice of forensic science.

You can read about the effort here:


NIST Names Members to First Forensic Science Standards Board

For Immediate Release: June 26, 2014

As part of its efforts to improve the scientific basis of forensic evidence used in courts of law, the U.S. Commerce Department’s National Institute of Standards and Technology (NIST) and the Department of Justice (DOJ) have made the first appointments to a new organization dedicated to identifying and fostering development and adoption of standards and guidelines for the nation’s forensic science community.

cartridge case mark
Forensic firearms and toolmarks: Members of the new Forensic Science Standards Board will coordinate development of consensus standards by committees dedicated to various forensic science disciplines, including firearms and toolmarks. Impressions made on the surface of a cartridge case when a gun is fired can act like fingerprints to identify a specific firearm.
Credit: NIST
View hi-resolution image

NIST and DOJ named 17 academic researchers and forensic science experts to the Forensic Science Standards Board (FSSB), a key component of NIST’s Organization of Scientific Area Committees (OSAC), which is bringing a uniform structure to what was previously an ad hoc system.

“The appointments to the Forensic Science Standards Board essentially mark a transition from planning to doing,” said NIST Acting Director Willie May. “After months of collaboration with the forensic science community, we are bringing to life this new organization that will have a positive impact on the practice of forensic science in the United States.”

The board will oversee three resource committees and five scientific area committees. Subcommittees will focus on specific disciplines, including DNA, toxicology, medico-legal death investigation, facial identification, latent fingerprints and firearms and toolmarks, among others. The subcommittees will propose consensus documentary standards, for adoption by the board, to improve quality and consistency of work in the forensic science community.

The establishment of the OSAC is part of a larger collaboration between NIST and DOJ, which announced the members of a new National Commission on Forensic Science in January 2014. A NIST-DOJ membership selection team is reviewing applications for all remaining positions in the OSAC and will announce the appointments as they are completed.

The new board includes five members who represent the research community, five members who chair the OSAC scientific area committees, six members who represent national forensic science professional organizations, and one ex officio member—Mark Stolorow, director of OSAC affairs for NIST.

The research community representatives are:

  • Joseph Francisco, Ph.D., William E. Moore distinguished professor, Purdue University;
  • Anil Jain, Ph.D., distinguished professor, Michigan State University;
  • Karen Kafadar, Ph.D., statistics professor, Indiana University; Department of Statistics professor and chair, University of Virginia; (after 8/26/2014)
  • Sarah Kerrigan, Ph.D., Forensic Science Department chair, Sam Houston State University; and
  • Douglas Ubelaker, Ph.D., curator, Smithsonian Institution, National Museum of Natural History, Division of Physical Anthropology.

Six members were nominated by their professional associations:

  • Andrew Baker, M.D., National Association of Medical Examiners standards committee chair and Hennepin County Medical Examiner, Minn.;
  • Laurel Farrell, Society of Forensic Toxicologists director and past president and American Society of Crime Laboratory Directors/Laboratory Accreditation Board instructor;
  • Steven Johnson, International Association for Identification first vice president and Ideal Innovators Inc. certified latent print examiner/facial examiner;
  • Mark Keisler, Association of Firearm and Tool Mark Examiners past president and member-at-large and Indiana State Police Laboratory Forensic Firearms Identification Unit supervisor;
  • Barry Logan, Ph.D., American Academy of Forensic Sciences past president and NMS Labs vice president of Forensic Science Initiatives and chief of Forensic Toxicology; and
  • Jeremy Triplett, American Society of Crime Laboratory Directors advocacy committee chair and Kentucky State Police Forensic Laboratory supervisor.

Five members will chair the OSAC scientific area committees (SAC):

  • Richard Vorder Bruegge – SAC IT/Multimedia; Federal Bureau of Investigation, senior photographic technologist;
  • Gregory Davis, M.D. – SAC Crime Scene/Death Investigation; University of Alabama at Birmingham, professor, division director, and chief coroner/medical examiner;
  • George Herrin Jr., Ph.D. – SAC Biology/DNA; Georgia Bureau of Investigation, Division of Forensic Science deputy director;
  • Austin Hicklin – SAC Physics/Pattern; Noblis, biometrics and forensic science fellow; and
  • Scott Oulton – SAC Chemistry/Instrumental Analysis; Drug Enforcement Administration, associate deputy assistant administrator.

In the future, board appointments will be for three-year terms. These first members will serve terms of two, three, or four years, to ensure continuity.

As a non-regulatory agency of the U.S. Department of Commerce, NIST promotes U.S. innovation and industrial competitiveness by advancing measurement science, standards and technology in ways that enhance economic security and improve our quality of life. To learn more about NIST, visit To learn more about OSAC, visit