The simple case for reporting of Uncertainty Measurement in measures in the Courtroom

This post is inspired by a combination of two events. First a comment by opposing counsel, and second several comments by several “old school” forensic science practitioners at this year’s American Academy of Forensic Science (AAFS) annual meeting that I attended. In a contested hearing opposing counsel argued that there was “no such thing as metrology.” He said this despite the fact that he heard from two metrologists about the well-established science, and its application. The second set of comments came from many at the AAFS meeting when Ted Vosk, Esquire who is a friend and colleague of mine was presenting at the meeting. He was lecturing at a part of a presentation on Uncertainty Measurement (UM) reporting in the forensic arena. In reaction to his words, some people commented that UM reporting was a “waste of time” or a “useless exercise.” One person commented that if it were to be done “where would I stop the figuring of UM.”

I actually think it is a simple case.

Quite frankly, I don’t understand what all of the hub-bub is about in not reporting UM.

What is the confidence of your measure? Are you a house cat who thinks he is a lion?
What is the confidence of your measure? Are you a house cat who thinks he is a lion?

Whereas my good friend Ted Vosk, Esquire made a very good, very convincing and very impassioned plea to the analyst’s sense of justice and science, I am going to try to be more practical. I am going to make an appeal to your logical bias.

Here is my open address to all of those involved in forensic science (regardless of whether you are employed by a prosecutor or a defender) in terms of UM reporting:

An opinion letter to all those in forensic science laboratories today

Dear Forensic Scientist,

I know you are not the robot that you claim to be when you preform some form of science. I understand that you are a real life human being. As such, you have bias. And you know what? Here’s a dirty little secret: it is totally acceptable that you do. You cannot not have bias. You have no choice in the matter. It is failing to acknowledge your bias that is dangerous. If you acknowledge you have bias, then you can take steps to mitigate it and try your best to not allow it inappropriately to influence your process, your procedure, your performance, your interpretation, your opinion and your conclusion.

Your bias could be as extreme as that you want one side to win. Your bias could be that you want to defend your interpretation or your opinion. Your bias could be that you want to defend your data. Your bias could be that you want to defend your profession. Your bias could be that you want to defend what you do or did.

To that end, I appeal to your bias with this. Logically.

1. You are not the finder of facts.

2. You are not supposed to be an advocate.

3. To do otherwise, you are an editor of facts.

4. When you present your measure as an absolute value, the old adage of “A half truth+ A half truth= A full lie” applies.

But you still say “Why present UM at all or why should I present it unless it is near a critical value?”

Well, it’s simple. These days, criminal defense lawyers win by exposing the whole truth when you chose not to present the whole truth. When you present a measure, whether it is a qualitative or quantitative measure, as an absolute and therefore free of any sort of doubt or error, you know scientifically this is wholly wrong.

A half truth+ A half truth= A full lie.

My colleagues are slowly learning the whole scientific truth. When you show half truth, we show the whole truth truth. We show the truth in the limitation of the assay performed, the truth about the limitation of your knowledge and experience, and the truth that you made assumptions or interpretations or judgment calls along the way.

No matter how much you try to justify on re-direct this initial lack of full disclosure of the whole truth, you will likely lose. Also, it frequently doesn’t matter if on re-direct examination if you have the UM ready to report. You have been exposed. There is doubt.

While the simple truth is in some tightly controlled and truly validated methods, the demand for honest and complete reporting in the expanded UM in both the quantitative measure and the qualitative measure (using acceptable metrologically acceptable methods such as the propagation of errors method or Monte Carlo analysis) may actually show that there is no possible way that the value could be below the critical measure, in my view, I say “Good,” and “So be it.” If you can legitimately and statistically prove (not just simply a stated value) that it takes 6, 7, 8 or 100 sigma to get below the critical value, then you have nothing to fear do you? But do you know you are in control?

If it is true, then that is what belongs in the courtroom and nothing else.

To do otherwise is a scientific sin (Vosk’s point) and will make you seem deceptive because you know what? You are (my point).

True science is not your private parochial sandbox that you need to “protect” us from, but rather it is for all of us to share in the joy of unbiased discovery of the truth at the temple of empircism.

With true sincerity as a true admirer of validated science,

Justin J. McShane, Esquire

Leave a Reply

Your email address will not be published. Required fields are marked *