Is it legitimate for a crime laboratory to use ‘historical data’ to prove its test results are valid?

I am lucky to get questions from colleagues. I am honored by it, and I thank them for the opportunity to answer their questions. From time to time I am going to feature questions that I get from them if they provide for teachable concepts.

This question from a colleague provides for just such an opportunity. Here is the question:

Hey, Justin. I have a DUI case where the lab did an analysis by Headspace Gas Chromatography (HS-GC-FID) and it turned out a result of 0.10 BAC. He is involved in an industry where he is occupationally exposed to EtOH all day long as well as other chemicals and Volatile Organic Compounds (VOCs). Despite the fact that OSHA requirements call for him to use a respirator, he doesn’t. After a long day working but not using the respirator, he had some beers (verified by point of sale records, video from the bar, and independent witnesses), but nowhere near enough to phenomenologically get him up to a 0.10 BAC. I was finally able to get the discovery after getting a judge to sign a motion to compel. I was looking at the data. In the data, within the run there was no separation or resolution matrix and no calibrators. I was confused so I called the lab, they sent me some more data. They told me that the lab had a policy of only doing a calibration curve once a month, and running a separation or resolution matrix of 5 molecules twice a year when the state sent them a proficiency test. This doesn’t sound right. Is it? Are there any resources such as peer reviewed studies or treatises that specifically says this is wrong?

In essence, my colleague is asking “Is it legitimate for a crime laboratory to use ‘historical data’ to prove its test results are valid?”

In our sister blog, we wrote a blog post about the lack of standards in the forensic science community. See generally, “Why forensic science testing for DUI BAC determination is the silly sister of analytical science: Good Laboratory Practices” and again in “A large problem in Gas Chromatography: No uniform standard for GC run position or composition.” So, we are in a really unusually position. As there are no standards in the community. It is like the Wild Wild West.

However, the three basic issues that jump out to me as potential issues with this method are as follows: (1) Qualitative measurement validity issues; (2) Quantitative measurement validity issues; and (3) the true source of the EtOH measure.

1. A very basic benefit and feature of Gas Chromatography (GC) is its power to resolve, and its potential power to be specific and not merely selective.

1.1 The difference between a test being merely selective and being truly specific would be the following example. If a test is selective, then it can tell the difference between compounds, but not specifically or uniquely. For example, enzymatic assay (hospital blood) is selective in that it reacts to EtOH but also reacts similarly to lactate and isopropanol (and there are many more compounds). So, in the case of enzymatic assay, if you have a sample with lactate, isopropanol and EtOH, the method cannot tell you the difference. It reports all of those present in whatever amount together as EtOH. This is a selective test. It is far from specific. This, in part, is why enzymatic assays are not forensically defensible. See generally,  “WellSpan is to be commended, but why people who have been convicted for decades of a DUI should be upset.” A specific test is one that can provide for uniqueness in its qualitative result, meaning that the result reported is from that specific compound alone and to the absolute exclusion of everything else. It is definitely from EtOH and nothing else, meaning no lactate or isopropanol contribution to the result.

1.2 If designed and performed perfectly, meaning HS-GC-FID, it has the potential to provide a unique qualitative measure (what compound do we have to the exclusion of everything else in the universe), and then IF AND ONLY IF we can separate do we have the potential to achieve a quantitative measure (how much do we have). The GC itself is a qualitative analytical method whereby if it alone is used, it can only tell that the characteristic is present by the retention time as compared against a known. At best, using only a GC, you can tell that the suspect analyte is consistent with the characteristics of the known. It does not necessarily mean that it is the analyte until you use a detector of some level of supreme analytical sensitivity such as TOF or MS.

1.3 If we cannot master the first prerequisite of obtaining a valid result which is separation, then we cannot validly quantitate. Therefore, it’s all about separation–the chromatography.

1.4 Separation, separation, separation.

1.5 Separation must be proven! It must be demonstrated within each batch (the group that comprises the run that includes the knowns-such as the controls, and the unknowns-such as the accused DUI drivers) on the day and within the run that contains the accused motorist’s results. In an ideal world, we would demand that everything that that person comes into contact with that may be in his or her blood be proven to be separated from the EtOH. If separation is not proven on the day and within the run to include all substances that may be in that person’s blood, then we violate all of the above. Separation first. Without separation we cannot validly quantitative. We fail.

1.6 Separation is proven by using CRMs in the manner outlined here: If he was an analytical chemist or a DUI defense attorney he would say “Show me the separation!!”

1.7 In this case, according to laboratory protocol, we have attempts to prove separation. But it is of inferior quality as it has only proved separation between methanol, ethanol, isopropanol, acetone, and n-Propanol. These are only 5 compounds out of the 65 million registered with Chemical Abstract Service (CAS) registration. Not that we should require all 65 million be separated by the assay, but the most common ones should be. This should include environmental exposure to other compounds (such as your case), and whatever was used to perform the cleansing of the venipunture site. If they do not provide for separation of the EtOH from these, then how can we exclude the possibility that the peak on the chromatogram is not unique to that compound and is not due to the contribution of some other compound? Remember selective versus specific. Remember proven separation first, and if and only if separation is proven, do we move on to quantification.

1.8 If separation has not been proven on your day and within your batch, and if for example, all of these chromatograms where a resolution matrix or separation matrix were from dates other than the date of the analysis of your unknown, this is violative of best practices in Good Laboratory Practices and in the regulated world such as EPA and FDA. So, they have not proven separation on the day and within the batch of the analysis of your unknown. The absolute most they can scientifically say is that they are presuming so. They have no data to support the conclusion that on the day and time of the testing of your unknown and under the chromatographic conditions that your person’s test was run on that they can prove and show separation. This is especially so when they cannot show separation of all matters that may be in the blood such as environmental exposure to VOCs. Remember, there should be no presumptions in science. In the courtroom, by law, there is but one presumption. That is not that the laboratory could prove separation because they did so as to some perhaps irrelevant compounds at some irrelevant time before or after, but that of the presumption of innocence.

Headspace Gas Chromatograph with Flame Ionization Detector
Headspace Gas Chromatograph with Flame Ionization Detector

2. A HS-GC-FID machine does not know what it is examining (the qualitative measure) or how much it is measuring (the quantitative measure) out of the box when it comes from the vendor that sells it. It has to be “trained” or “taught.” The first part of “teaching” it what it is examining (the qualitative measure) is done through the resolution or separation matrix described above. The second part of training the machine how much is through the act of calibration (described below).

2.1 We can tell how much we have (quantitation) only after we separate. (I know I am beating a dead horse, but I need to impress this upon you). Quantitation is achieved through establishing a calibration curve. Without a demonstrated and proven calibration curve as to both the target analyte (EtOH) and the ISTD (typically n-prop), then you absolutely cannot quantitate validly. More on the importance and the nature of calibration curves can be read here: “When is a straight line a curve: Calibration curve” and “How do they make the squiggly lines turn into a magic number: Area under the peak.”

2.2 If quantitation has not been proven on your day and within your batch, and if for example, all of these chromatograms and the resultant calibration curves were from dates other than the date of the analysis of your unknown, this is violative of best practices in Good Laboratory Practices and in the regulated world such as EPA and FDA. So, they have not proven quantitation on the day and within the batch of the analysis of your unknown. The absolute most they can scientifically say is that they are presuming so. They have no data to support the conclusion that on the day and time of the testing of your unknown and under the chromatographic conditions that your person’s test was run on that they can prove and show proper quantitation. Remember, there should be no presumptions in science. In the courtroom, by law, there is but one presumption. That is not that the laboratory could prove quantitation because they did so at some irrelevant time before or after, but that of the presumption of innocence.

3. Source issues. As much as I love HS-GC-FID, there is a big limitation of it.

You cannot always tell just by looking at the blood tubes if there are issues

3.1 It is crucial to note that even the most perfect method and the most perfect analysis by the most perfect laboratory on the most perfect machine cannot determine THE UNIQUE SOURCE of the result. Any test that is per­formed is only as good as the sam­ple. Even if the laboratory does the mythical perfect analysis, if the sample is not representative of the true condition of the blood as it was flowing in the person, then the reported result can never be cor­rect (true). Put more simply “Garbage In=Garbage Out (GIGO).” What I mean by that is that laboratories work on the Garbage-In-Garbage-Out principle.

3.2 Here is some more information on the GIGO principle: “Retesting Blood Alcohol Results

Garbage In=Garbage Out
Garbage In=Garbage Out

3.3 It is the difference between endogenous (that which is your client’s fault from consumption and existed in the tube at the time of test collection) EtOH and exogenous (that which is from outside of the natural organic state) EtOH conttribution (environmental pre-exposure of some substance that the method has not demonstrated separation, neo-formation of EtOH through putrification due to Candida Albicans or alternative source such as introduction through an IV line of some substance that the method has not demonstrated separation from the target analyte).

3.4 As scientific  skeptic, which the fact-finder must be, we should form the hypothesis that the sample itself  is garbage and seek tests that can confirm or dispel this notion (falsification).

3.5 The issue in Blood Alcohol Content measurement of the GIGO issue manifests itself in environmental pre-exposure of some substance that the method has not demonstrated separation, neo-formation of EtOH through putrification due to Candida Albicans or alternative source such as introduction through an IV line of some substance that the method has not demonstrated separation from the target analyte.

3.6 Garbage in=garbage out.

To see other posts on HS-GC-FID, please visit our Gas Chromatography section at www.PADUIBlog.com and from this blog we have:

2 Responses to “Is it legitimate for a crime laboratory to use ‘historical data’ to prove its test results are valid?”

  • Dustin Barr says:

    I aplogize if this sounds combative, I don’t have much finesse when it comes to writing. I understand the shortcomings of what you suggest when using GC-FID to quantify, but it could be misread to suggest that quanitation with GC-FID is never reliable since we can’t prove that the analyte in question was separated from every other compound in the world. The fact of the matter is that not all compounds do separate with CG-FID as I’m sure you already know. I will admit that toxicology is not my forte, but it’s misleading to suggest that quantitation is not reliable because you can’t prove that you are only looking at the analyte of interest. In my line of work, quantitation is never enough. We must confirm the separation of the analyte by either GC-MS or GC-IRD. Since this test is specific, it can be concluded that what separates on GC-MS also separates on GC-FID.
    Concerning running standards with every batch, you have to have an appreciation of what is expected in a foresic environment. Americans have right to a speedy trial and if we had to perform all analysis to GLP standards, I assure you the forensic backlog would be insurmountable. I’m not making excuses for shotty analysis, because I don’t believe, first of all, that it is shotty, but having worked in both a GLP lab and now in a forensic lab, I can speak at length about the different requirements and why they exist in both.
    Here is an example. In GLP labs, you must prove the extraction is accurate with QC recoveries in identical matrix. You must also run a range of standards in the calibration curve and if the sample doesn’t fall in that range, then it is not usable. The calibration curve and recovery QC’s must be run at the same time as the samples. This is impractical in the analytical forensic chemistry world because a typical GC run takes up 12 minutes and my batches in the GLP world typically contained more than 50 samples.
    Now here is why we don’t need to perform to GLP standards in the forensic realm. As I already stated, it is not necessariy to prove separation on quantitation if you have proven it elsewhere. Furthermore, running a calibration curve with the batch is not required by ASCLD/LAB. What is required is that the quatitaion method must be validated once and verified monthly. We run a verification standard with each of our batches and it doesn’t seem like this was done in this case. I would be confident with my analytical results without the verification standard if the method verifications done monthly were good both before and after the analytical acquistion.
    I don’t know why I felt like I should respond, I just wanted to add a little more clarity.

  • I appreciate all perspectives and points-of-view. I want this to be a place of discourse. Please encourage your colleagues to visit this site and engage with all of us. It makes us all better. I thank you for it.

    You make some points.

    Working in the DEA laboratory must provide for a unique perspective.

    Here are some of my simple thoughts.

    1. Why on Earth do we require a more rigorous process to bring aspirin to the market than to convict a person?

    If we use non-specific means of qualitative measure (GC-FID), then how can we be sure beyond a reasonable doubt when quantificaiton matters? (think blood alcohol)

    2. While I agree that it is not necessary in the typical analysis to separate out all 65 million registered compounds, but certainly more than 2-6 which is what I typically see in laboratory work that gets reported out and presented in trial in the United States.

    3. Maybe shocking to you, but I think you and I agree as you make my main point which simply is that no one method of instrumentation can provide the whole picture. There needs to be a true orthogonal approach. If you need non-complementary methods of analysis to patent a drug, then why don’t you need the same to identify it when it is in blood?

    4. If you, as an experienced chemist, would not be allowed to bring the aspirin to the market using the process that you now employ in the forensic laboratory (as you would be fined and shut down by the FDA), why can you try to deliver “justice” using those methods?

    It just doesn’t make sense.

    5. The issue of limited resources, in my (not trying to be combative either) opinion, is a really a non-excuse. I think you folks need more resources. A lot more. I lobby for it.

    In what I see of it, the backlog issue is due in a large part to really what I see as mismanagement by the prosecuting agency when they don’t tell you what cases have plead out and/or fail to exercise any sort of common sense discretion over the contextual need for analysis (the “we have to get DNA off everything including his own toothbrush” mentality).

    While I appreciate and understand the demands the system places on analysts such as yourself, at the end of the day if you are not being provided with what you need to be sure and to be truly scientific in your approach towards qualitative and quantitative analysis such as you were in your pharmaceutical GLP lab, then you and your colleagues can refuse to participate in it. No one is forcing anyone to work in a laboratory. You can say no. I say no to cases all day long.

    [As an aside, if ASCLD/LAB (even their International program) is a measure of “good science” to you, then you and I will simply have to disagree. While ASCLD/LAB is certainly better than nothing far too many labs try to point to it as a shield to fend off review and present it as somehow meaning that they are infallible.]

    Why not be open to scientific verification?

    Look, I win cases, not typically because of some major screw up in the testing (although that happens more often than anyone would like especially in local and state laboratories), but because there is a lack of robust universal standards that are based in basic science employed. Non-validated methods are used. Basics of validation are ignored.

    I expose this. It doesn’t look good.

    Further, there is a culture of trying to make data impossible for the defense to try to get. For example, would your laboratory openly give me an uncontrolled copy of your the current policies, procedures and instructions for the analysis that you do every day? I doubt it.

    I expose this. It looks like the laboratory has something to hide. Jurors and judges don’t like that.

    On the other hand, folks overstate the conclusions of their testing by constantly presenting it as absolute (especially in the world of quantification). They overstate their own knowledge. They fail to acknowledge the limitation of the assay that they use. They fail to expose the limitation of their own knowledge. They become salesmen or saleswomen.

    They come into court with one piece of paper that has a conclusion of their analysis with no underlying data and basically say “trust me.”

    I expose it. It doesn’t look good. It casts doubt.

    Simple solution. Employ a more robust method that is scientifically based. Be open. Just be straight up.

    I am open to reading about how I am totally wrong.

Leave a Reply

Your email address will not be published. Required fields are marked *