To inspire fierce and vitriolic debate in the legal-forensic community, all one has to do is to mention the name Cameron Todd Willingham.

We have blogged on the case and other arson cases before:

In full disclosure this office and myself personally am litigating a “bad arson science case.” You can read about it here:

Mr. Hugney’s website is here:


I read The Marshall Report on Cameron Todd Willingham the minute it came out. I also read the entire Grievance that was filed against the former prosecutor of the Willingham case: John Jackson. I understand and appreciate that the grievance is one side of the story. I also understand Jackson, who later became a judge and has retired from law in 2012, had vehemently denied that there was any sort of deal even suggested in exchange for the testimony of the jailhouse snitch in the case: Johnny Webb. Now as is reported, Jackson is not denying trying to help Webb after the Willingham conviction (something that he said that he could not and would not do at the Willingham trial). It would be very hard for him to deny trying to help Webb given the letters that he wrote on Webb’s behalf. I understand that it has been reported that Jackson freely admits that he did so, but for reasons not having to do with a secret undeclared deal. There is a paper trial that according to the Washington Post and The Marshall Report includes:

  1. An unsigned note in the files of the Navarro County District Court says that Jackson asked the clerk to clarify that Johnny Webb’s conviction was not for aggravated robbery, but for a lesser charge.
  2. On Oct. 21, 1992, days after the note to the clerk, Jackson sent a letter to prison officials requesting that Webb be assigned to a medical unit, which would be less onerous than protective custody. “Mr. Webb was a pivotal witness in a capital murder prosecution,” Jackson wrote. Webb had “placed himself at risk based upon his testimony in the case and I fear that he may suffer reprisal if placed in the general population.” He added, “Webb’s testimony may be necessary at (a) later stage of the proceedings and I would appreciate your attempting to place him in an environment that guarantees the smallest risk.”
  3. Then unbelievably, like manna from heaven for John Webb, on July 15, 1996, at Jackson’s request, Judge Kenneth “Buck” Douglas — who had presided over Willingham’s trial and sentenced Webb to prison in 1992 — entered a new judgment in Webb’s case. The crime was officially recorded as a conviction for second-degree robbery instead of an aggravated robbery, in effect reducing the time Webb was required to wait before seeking parole.
  4. Jackson followed up with a letter to the head of the parole board saying Webb “volunteered information and testified … without any agreement from the State respecting diminution of the recommendation in his own case.” He asked that Webb be given consideration for his “Cooperation in the murder prosecution without expectation of leniency.”
  5. A month later, Jackson followed up with another letter requesting that Webb be transferred back to the Navarro County Jail because Webb had received death threats from other inmates. “In the event of a reversal,” Jackson wrote, “I would also like to be able to count on Webb’s continued cooperation.”
  6. Webb requests to be transferred from state prison to the Navarro County Jail. “I kept my end of the promise,” he writes to the judge, “the state is bound to uphold theirs’ until my release from incarceration.”
  7. A letter to Webb from Jackson that states that he will use “…any reasonable means available to me to shorten your incarceration will be explored.”
  8. With Webb threatening to recant his testimony, Jackson wrote to Webb in 2000 to describe how he had worked with Pearce to get him released from prison early.
  9. And Webb writing to formally recant his testimony, he writes “I was forsed (sic) to testify against Mr. Willingham by the D.A.’s Office and other officials. I was made to lie. Mr. Willingham is innocent of all charges.”
  10. Incredibly, Jackson, a prosecutor, filed for clemency for Webb.
  11. Shortly after Jackson’s election as a judge in 1996, he organized another flurry of letters to the parole board. This time, Judge Douglas, Sheriff Cotten and Jackson all insisted that Webb’s sentence was excessive and he should be released. Upon ascending to the bench, Jackson issued a warrant to prison officials requiring Webb be brought to Navarro County for a hearing relating to “protective custody pending executive clemency.” No court records have been found showing that such a hearing was held.


Of course, a large amount of the details and the context comes from information from two sources: Johnny Webb, who has a host of crimen falsi and has variously recanted and supported his testimony through the years, and the now embattled former prosecutor turned judge who has had some arguably inconsistent statements of his own in this matter, John Jackson. But there are these letters and documents and support that there is at the very least an issue of continuing disclosure.


Also life experience has taught me that the prosecution only calls in a jailhouse snitch to testify if there is a large gaping hole in the case, and in their judgment an acquittal is likely without that testimony. It is an axiomatic fact. Otherwise, they would not risk calling the jailhouse snitch at all. And to call the jailhouse snitch as the first witness of the capital murder trial just emphasizes the importance of his testimony to the prosecution. All of these things factor into some new thoughts on the issue, at least for me.


I struggled with whether or not it was newsworthy to this forensic science focused blog. After sitting on it and thinking about it for several days, I have concluded that it is perhaps relevant. When it comes to court, right or wrong, there is never a clear delineation between objective evidence and subjective evidence and forensic evidence versus witness statements. One is often used to influence the other. One is often used to corroborate the other.

It seems that in these “change of science cases” what ends up happening is that the ancient notions that were once presented as legitimate science that have now been thoroughly debunked are not defended by the prosecution when modern claims come up. Instead, what they do is rely on “sliming up” the character and past moral failings of the convicted as a means of some sort of super indirect circumstantial evidence of guilt. The refrain in reading the ardent and passionate defenses to the execution of Cameron Todd Willingham include pointing to his verbal and physically abusive past, the allegations of trying to kill is children before the fire, and also the very last sentence of his life. While perhaps to some people, these may make emotional sense in the fact that bad guy is capable of doing bad things, that is not at all objective proof. If the science is not there, then how can their be a crime.

These are my personal previous thoughts on the issue.

It has been said that science is evolutionary and rarely revolutionary and that in the long enough term that science is self-correcting. With the heightened scrutiny being placed on several disciplines of forensic science leading to revisions of classically held but now severely questioned or abandoned notions, the question becomes what becomes of the convictions in cases that are decades old that depended on these now questioned practices. From fire debris to hair analysis to pattern recognition based disciplines, testimonies that are decades old and formed the lynch pin of convictions that feature either over-expressions of the interpretation of the analysis performed or are based upon now questioned practices are may require re-examination.

One of the cardinal principles in the law is that at some point judgments must become final and therefore unreviewable. The idea behind this is that litigants cannot continually race to the courthouse to continually re-examine cases. From a point of view of societal resources and economy this may make sense; however, in the case of justice and liberty, does it? What do we do when there has been a change of science? Should those who are clearly time barred by a matter of decades from resorting to the courts be able to now re-petition the court when there has been a sea change in the fundamentals of the underlying science that formed the conviction? Should those who have served their sentence or have paid society’s ultimate price in that the death penalty was executed have the investigation into potential injustice shrugged off as a matter of no consequence?

One study cites 10,000 innocent people are convicted each year in the United States. Even if this number is overstated by as much as 100%, simple math reveals that the number of innocent people behind bars is staggering. Some states have very simple time bars that require all claims including change of science cases be alleged within a year of conviction or within 60 days of discovery. Other states, such as Texas, are beginning to allow writs when there are claims of innocence even with decade old convictions. But what happens when the sentence is served? Resort to the federal courts to overcome these no longer scientifically supported convictions frequently run into the Antiterrorism and Effective Death Penalty Act of 1996. One provision of the AEDPA limits the power of federal judges to grant relief unless the state court’s adjudication of the claim resulted in a decision that was (1) contrary to, or involved an unreasonable application of clearly established federal law as determined by the Supreme Court of the United States; or (2) based on an unreasonable determination of the facts in light of the evidence presented in the state court proceeding.

All of this strikes at the heart of what is fair and just. What do you think?


NIST names OSAC Resource Committees

As many regular readers will recall, NIST is trying to end the Wild Wild West that is the modern practice of Forensic Science. Just moments ago, they released their latest appointees:

An initiative to strengthen and bring uniformity to forensic science standards took another step forward today as the National Institute of Standards and Technology appointed 35 new members to the Organization for Scientific Area Committees (OSAC).

The new members, selected for their expertise in law, psychology and quality assurance, will serve on three advisory committees. These OSAC Resource Committees will play a critical support role by advising the Forensic Science Standards Board, the scientific area committees and subcommittees focused on specific forensic science disciplines within OSAC as they adopt, develop and review standards.

“As our science-focused committees and subcommittees work to support the development of forensic science standards and guidelines, we expect that there will be many questions related to law, work flow processes and quality control. These resource committees will help address those,” said John Paul Jones II, associate director for OSAC affairs.

The Human Factors Committee will provide guidance on how systems design influences human performance, on how to minimize cognitive and confirmation bias, and on how to mitigate errors in complex tasks.

The Legal Resource Committee will review and provide a legal perspective on proposed standards.

The Quality Infrastructure Committee will assemble and update a Forensic Science Code of Practice, provide guidance on quality issues, and provide impact statements that inform agency management about how specific standards may affect laboratory operations. It will also work with outside standards development organizations and accrediting bodies as needed.

The resource committee members were chosen from among 1,300 OSAC applicants. They include public defenders, law school professors, prosecutors, judges, standards development experts, laboratory managers and human factors experts.

A NIST-DOJ membership selection team is reviewing applications for the remaining OSAC positions and will announce the appointments as they are completed.

To see the membership of each resource committee, please go to


Human Factors Committee Members

  • Deborah A. Boehm-Davis, Ph.D., Dean of the College of Humanities and Social Sciences, George Mason University
  • Itiel Dror, Ph.D., Principal Researcher, Cognitive Consultants International
  • Cleotilde Gonzalez, Ph.D., Associate Research Professor of Decision Sciences, Carnegie Mellon University
  • Christian A. Meissner, Ph.D., Professor of Psychology, Iowa State University
  • Erin Morris, Ph.D., Behavioral Sciences Research Analyst, Los Angeles County Public Defender
  • Sunita Sah, Ph.D., Assistant Professor, Strategy, Economics, Ethics and Public Policy at Georgetown University McDonough School of Business
  • Scott Shappell, Ph.D., Human Factors and Systems Department Chair, Emory-Riddle Aeronautical University
  • Dan Simon, Professor of Law and Psychology, University of Southern California, Gould School of Law, and Department of Psychology
  • Brian C. Stanton, cognitive scientist, National Institute of Standards and Technology
  • William C. Thompson, Ph.D., Professor of Criminology, Law, and Society and Psychology and Social Behavior and Law, University of California Irvine

Legal Resource Committee Members

  • Jennifer Friedman, Deputy Public Defender, Los Angeles County
  • Christine Funk, General Counsel, Department of Forensic Sciences, Washington, D.C. (local government)
  • Lynn Robitaille Garcia, General Counsel, Texas Forensic Science Commission (state government)
  • Ted R. Hunt, Chief Trial Attorney and DNA Cold Case Project Director, Jackson County Prosecutor’s Office, Kansas City, MO
  • John Kacavas, United States Attorney, U.S. Department of Justice
  • David H. Kaye, Professor, Graduate Faculty, Forensic Science Program, Pennsylvania State University
  • David A. Moran, Professor of Law, University of Michigan Law School
  • Christopher J. Plourd, Superior Court Judge, State of California
  • Ronald S. Reinstein, Judge and Judicial Consultant, Arizona Supreme Court
  • Barry Scheck, Professor, Cardozo School of Law, Yeshiva University; Co-Director, Innocence Project; Commissioner, NY Commission on Forensic Science; Neufeld, Scheck, & Brustin, LLC

Quality Infrastructure Committee Members

  • Karin Athanas, Program Manager, American Association For Laboratory Accreditation
  • Sally S. Aiken, Medical Examiner, Spokane County, Washington
  • Barbara E. Andree, Forensic Chemist, Bureau of Alcohol, Tobacco, Firearms and Explosives
  • Jason Bond, Quality Assurance Coordinator, Indiana State Police Laboratory Division
  • Pamela L. Bordner, Sr. Accreditation Program Manager, ASCLD/LAB
  • Kris Cano, Forensic Laboratory Manager, Scottsdale Police Department Crime Laboratory
  • Deborah Friedman, Criminalist III, Broward Sheriff’s Office Crime Laboratory
  • Matthew Gamette, Laboratory Improvement and Quality Manager, Idaho State Police Forensic Services
  • Keith Greenaway, Vice President, ANSI-ASQ National Accreditation Board
  • Arlene Hall, Commander, Illinois State Police, Division of Forensic Services
  • Bruce Houlihan, Director, Orange County Crime Laboratory/Orange County Sheriff-Coroner
  • Alice R. Isenberg, Ph.D., Section Chief, FBI Laboratory
  • Timothy Kupferschmid, Laboratory Director, New York City Office of Chief Medical Examiner
  • Karen Reczek, Senior Standards Information Specialist, NIST Standards Coordination Office
  • Frances E. Schrotter, Sr. Vice President and Chief Operation Officer, American National Standards Institute

The problems with the modern practice of forensic firearm and toolmark analysis

The field is technically called a forensic firearm and toolmark analysis whose experts use an optical comparison microscope to examine the exemplar with the unknown recovered from the crime scene by a comparison criteria that is referred to as  “sufficiently consecutive matching striae” in an attempt to determine unique source.


Which on is it from? Can science really tell?

Which one is it from? Can science really tell?

The modern practice of forensic firearm and toolmark analysis leaves a lot to be desired. A lot. All pattern recognition endeavors do as well (e.g., latent fingerprint, handwriting, hair comparison, odontology-bite mark, lead bullet). It was subject to a few blog posts here not too long ago


The notion that with no validated and standardized criteria that have been statistically proven to provide for uniqueness that an examiner who has no formal tribology background can bridge the large gap between the millions of firearms in that class down to not simply a subclass and not merely to a batch in a given day, but rather to a specific firearm that is exclusive of all other firearms on the planet earth is pure apocrypha. Fantasy really.


First we have to understand their historical method and their historical training:

Here is how they are typically trained to look at the casings (note that it is in 2D)


Basic nomenclature used by them:

  • Striations: Lines or scratches left by the irregularities on the end of the tool as it scrapes away metal.
  • Stamped or Impression markings: Toolmarks created by the striking of a cartridge primer by the firing pin, or the head of a cartridge case by the ejector.
  • Stamped Striation Marks: Impressions left by the firing pin, ejector, and breechblock, if these parts have been manufactured by filing, or turned on a lathe.


In very brief this is what they are trained to do:

They test fire several rounds of ammunition of the same brand as crime scene ammunition through the suspect firearm. This makes the exemplars. They examine and compare test firings first. If they “match” perfectly, they examine the evidence recovered from the crime scene.

They begin their examination by first aligning the ejector and extractor marks properly. Then examine the firing pin impressions, noting the shape, position, depth and width. Then look for irregularities such as tips or depressions within the firing pin crater as a whole. Finally, look for identifiable machine toolmarks in the firing pin crater. Then examine the breech face marks, making notations regarding the location, length, width and contour of toolmark impressions. Extractor and ejector marks can also be examined and compared in a similar manner.


Here are straie in case you have never seen them:



There are many issues with this method of analysis and in particular its interpretation:

1. This is not the 19th century. Firearms are now mass-manufactured, thus eliminating much of the “uniqueness” of a gun that came from being built by hand back then. There is very little hand filing if ever these days. The firearms are made using cutting tools made of High Speed Steel (HSS) (e.g., T-1, M-2, M-7, and M-42) or Carbide (made of Tungsten Carbide based alloys) such as the below:


Modern cutting tools do not wear anywhere near as rapidly as they once did. They are very rugged as they are constructed or coated with cobalt, or Titanium Nitride (TiN) which is added to HSS tools to increase hardness. Titanium Carbon-Nitride (TiCN) is added to HSS or carbide tools to increase hardness and prevent metal from adhering to the tool. Titanium Aluminum Nitride (TiAlN) increases tool life during high temperature cutting. Polycrystalline Cubic Boron Nitride (PCBN). So, if the tools that make the toolmarks in the firearm last longer and fail to deform more readily, by necessary implication the difference from firearm to firearm is less distinctive.

This all involves is the beautiful science known as tribology. In tribology there is an entire fascinating filed that involves the geometry of chip formation and Built-up edge (BUE formation) and the shear fracture left behind from the use of a tool such as this:


2. The core of the issue of real world identification is that as you fire a firearm each time it will “wear” down the barrel. Metal on metal under gas pressure will do that. This concept is well known in the true forensic science community of tribology where the literature is overwhelming that an identification of source is simply not possible if the firearm is within its first 300 or so discharges through the barrel. How does the government know that a given firearm is inside or outside of its first three hundred discharges? They don’t, they just assume. Or they don’t know this key fact.

3. There is also the concept of how precisely and under what conditions there can be toolmark transference. Toolmarks are created by metal cutting tools on the machined surfaces of firearms. These toolmarks are then transferred onto the cartridge case surfaces by the expansion of the cartridge brass in the barrel chamber, and breech face of the pistol slide. In the case of bullets, toolmarks are transferred onto the bullet surfaces by the obturation of the bullet into the rifled barrel lands and grooves. The modulus of elasticity of typical cartridge brass is 16×10^6 PSI, and the modulus of elasticity for typical gun steel is 29×10^6 PSI. Obviously the lower modulus of elasticity of brass causes the brass to expand under high pressure into the barrel chamber, and pistol breech face. Modulus of elasticity is defined as the ratio of the applied stress or force to the change in shape of an elastic body. NOT ALL TOOLMARKS TRANSFER. There is also the issue of frictional forces between the moving parts that can leave marks as well.

4. The interpretation is not standardized and non-validated. The simple point in fact is that despite what we in the lay world believe there is infrequently any criteria (e.g., 12 points of comparison) to make this all important call from class to uniqueness.

  • Class Characteristics: Marks that all firearms of a given make and model will leave. These are coarser toolmark impressions.
  • Subclass Characteristics: Fall under this category, are present in only some toolmarks. They are created during batch lot manufacturing of cutting tools, and are present on only those tools from a specific batch lot. These are also coarser toolmark impressions.
  • Individual Characteristics: Marks specific to only one firearm. These are finer toolmark impressions.
  • Accidental Characteristics: Non-reproducible toolmarks that may appear on one spent cartridge case, but not on any other spent cartridge case.

Again, no universal standards exist delineating these characteristics. If there are any, they are not statistically sound in terms of their validity.

Further, traditionally, breech face marks and firing pin impressions on a cartridge case head (and land and groove impressions on a bullet) are considered individual marks. But, not all breech face marks or land and groove impressions are in fact unique to that gun. Studies show that up to 20% matching striations may be all unique to that gun, but all common to several guns of the same make and model, or a combination of both. Again, there are no standards to determine which breech face marks (or land and groove impressions) are unique to that gun. Firearms examiners routinely just assume that any matching breech face marks identify the gun uniquely when looking at them in 2D.

5. The real problem for this science lies in the cognitive bias inherent in the examination itself. You, the examiner, fire a bullet from a known firearm. You retrieve it. You examine the seized evidence round that comes from a police investigation. Boy. Do you think that you are primed for cognitive bias?

Here is a case study of where the prosecution through PSP first examined a recovered firearm and called it a “match” to seized evidence. They had no photomicrographs, just an opinion by a high school graduate. It was reduced to this report. Note that it is specific as to source.

10 When challenged, they then went and took the unusual step of making photomicrographs off of their comparison microscope that resulted in this. Note it is in 2D (Of course, it was still considered a “match”) When pressed with precisely how they came up with this theory of uniqueness, it all came down to the most horrible phrase ever uttered in a courtroom “based upon my training, knowledge and experience.”

11 The top is the recovered evidence and the bottom is the exemplar from the recovered gun. Note they are not even at the same magnification or in the same orientation.

Instead, we used a Scanning Electron Microscope (the below results are from this same case and are of the same evidence and exemplar where PSP said that it was a “100% match”) was revealed in 3D by SEM to be far from it as seen below when examined in 3D and topographically examined when just looking at the firing pin position and characteristics in 3D alone. (Luckily for us in this case, this examiner had previously testified in another case that was provided to me by a PACDL member that if the firing pin position and characteristics were not a “match” then it was a “positive exclusion” of the firearm.)

20 21 22

6. All of this is why there is a big push for funding the studies to consider the issue of at what point do the similarities become enough to distinguish and form valid opinions of specificity. When studies involving white light inferometers were used to get a 3D image (instead of the 2D image that the comparison microscope uses), the revealing part was “that only 21% to 38% of the striae on pairs of bullets fired from the same” gun “matched.” Wow!


Here is a data sheet from one result:

31 Instead, what do we do? Often non-credentialed lay witnesses who were former traffic cops who are trained by others similarly lacking in an engineering or tribology background using a quasi-apprenticeship method, make the call of a “match.”  The term of art (or the call meaning the use of the word “match”) used by these practitioners is arrived at by these untrained microscopist looking with what amounts to their “highly trained set of eyeballs” at a three-dimensional object under a insufficiently magnified stereo comparison microscope which is a two dimensional representation to arrive at the conclusion that there is supposedly “sufficient consecutively matching striae” to warrant a conclusion that the unknown and the known are in fact a match.


Delaware State Police (“DSP”) and the Delaware Department of Justice (“DDOJ”) initiated an investigation of the Office of the Chief Medical Examiner (“OCME”) Controlled Substances Unit (“OCME-CSU” or “CSU”).

According to this news story, there were lots of problems found.

New details have emerged in an evidence scandal that occurred within in the Delaware Medical Examiner’s office.

A new report from Attorney General Beau Biden’s office revealed numerous “systemic operational failings” within the Chief Medical Examiner’s Controlled Substance Unit. These failings allegedly led to missing or altered drug evidence in 46 cases.

An audit discovered at least 51 pieces of potentially compromised evidence. The missing evidence includes marijuana, Oxycontin, heroin and cocaine.

The preliminary investigation, conducted by the Department of Justice and Delaware State Police, demonstrated the absence of management, oversight and security within the lab. Detailing lax security procedures, employees were described in the report as propping the drug vault door open and turning off the security alarm system.

“Employees recall having observed the door to the drug vault propped open numerous times over the years,” stated the report. “When the DSP secured the drug vault on February 20, 2014, a well-worn, wooden chock was observed in the area adjacent to the door. Based on witness interviews, investigators believe this was used to hold the door open.”

At times, drug evidence was not handled, stored or tested according to protocol, according to the investigation. Records were mismanaged and evidence was removed without being properly logged out. Additionally, the report stated that some employees lacked the training or experience needed to perform some of the tasks to which they were assigned.

The investigation has lead to the arrest of two lab employees, Forensic Investigator James Woodson and Laboratory Manager Farnam Daneshgar, along with the suspension of Chief Medical Examiner Dr. Richard Callery.

Woodson was indicted on one count each of trafficking cocaine, theft of a controlled substance, official misconduct, and tampering with evidence.

Daneshgar was indicted on two counts of falsifying business records. According to the report, witnesses accused Daneshgar of “dry labbing,” which is the “practice of declaring a result without performing the analytical testing to produce the result.”

Overhaul Planned

The report was released as Delaware lawmakers prepare to vote on a bill that would abolish the chief medical examiner’s office and create the Division of Forensic Science. A director would oversee the division, which would be housed under the Department of Safety and Homeland Security rather than the Department of Health and Social Services.

The division would be responsible for overseeing the chief medical examiner’s office and would be responsible for working with the courts and law enforcement, investigating deaths, participating on the Criminal Justice Council and providing fatal incident reviews to the Domestic Violence Coordinating Council.

“Delaware must have its own independent, state-of-the-art crime laboratory,” said Biden in a statement. “A new crime lab is the right thing for Delaware’s criminal justice system and the right thing for taxpayers.”

The legislation is being considered in the state Senate.