In a series of posts, we are going to talk about method validation.

1. Part 1: Introduction-Is it valid, invalid or non-validated?
2. Part 2: What is method validation?
3. Part 3: Can we use some­one else’s val­i­dated method?
4. Part 4: What trig­gers ver­i­fi­ca­tion, re-validation or out right new val­i­da­tion of a method?
5. Part 5: What are the essen­tial terms in method validation?
6. Part 6: What is qual­ity assur­ance and qual­ity control?

As we dis­cussed in our last post,

Qual­ity Con­trol: Pro­ce­dures which give insight into the pre­ci­sion and accu­racy of analy­sis results. The main­te­nance and state­ment of the qual­ity of a prod­uct (data set, etc.) specif­i­cally that it meets or exceeds some min­i­mum stan­dard based on known, testable criteria.

Qual­ity Assur­ance: The guar­an­tee that the qual­ity of a prod­uct (ana­lyt­i­cal data set, etc.) is actu­ally what is claimed on the basis of the qual­ity con­trol applied in cre­at­ing that prod­uct. Qual­ity assur­ance is not syn­ony­mous with qual­ity con­trol. Qual­ity assur­ance is meant to pro­tect against fail­ures of qual­ity control.

Won­der­ful. We can read, but what does this mean?

When­ever I think of qual­ity con­trol, I think of two things: (1) the res­o­lu­tion standard/separation matrix, and (2) the cal­i­bra­tion curve. What I am talk­ing about is the heart of any ana­lyt­i­cal chem­istry process: (1) the abil­ity to be spe­cific, and (2) the abil­ity to weigh. Qual­ity Con­trol is the designed sys­tem that is tested and found to be suf­fi­cient to pro­vide use­ful and “good” data that will sup­port the con­tention that the machine is indeed capa­ble of pro­duc­ing the results the ana­lyst and the method wants. It is not a sys­tem of sim­ply test­ing things in ran­dom order, but is a uni­form repeated process that has been proven to pro­duce data that sup­ports the goals of the test­ing. This is very impor­tant. We need to test all unknown sam­ples as part of a designed and delib­er­ate process that as a sys­tem is designed to pro­duce valid results. We need to not only have instruc­tions (e.g., place 4 cal­i­bra­tors of con­cen­tra­tions of 0.02, 0.08, 0.16 and 0.40 from cer­ti­fied ref­er­ence mate­ri­als in vial posi­tions 2, 3, 4, and 5 within the run/batch) but we have to have data that sup­ports that doing so in this exact order and in this exact way pro­duces and insures valid results and pro­tects against invalid results.

This is where a lot of foren­sic sci­ence crime lab­o­ra­to­ries fail. They have not con­ducted exper­i­ments into the true rugged­ness of their process. In a rugged­ness study (some­times referred to as a robust­ness study), we change vari­ables and inputs to see if it changes the results. Another way of look­ing at it is that we look to opti­mize (stream­line) the method and see when or if the method reaches a “break­ing point.”

Is our method del­i­cate or is it more robust/rugged?

Every machine needs to be “taught” what to do and how to do it because out of the box, each machine is a dumb device. In the case of blood ethanol test­ing for exam­ple, the ana­lyst has to teach it what is ethanol and most impor­tantly what is not ethanol. This is not as easy as it seems as there are over 65 mil­lion reg­is­tered chem­i­cal com­pounds accord­ing to CAS. If you don’t teach it right, mean­ing how to pick out ethanol and ethanol alone to the exclu­sion of every other com­pound in the uni­verse, then you will get the wrong result. It can tell us how much, but only of what it selected out. So, if you did not teach the machine right as to how to pick out ethanol uniquely, then when it next moves on to weigh­ing stuff, it will report the wrong result. The machine is a fancy bean counter, but you have to show it what to count and what not to count in order for the result to be right.

Qual­ity Assur­ance on the other hand has two basic lev­els. It is basi­cally a “dou­ble check” mech­a­nism and also an error report­ing mech­a­nism, if set up cor­rectly. We should expect ana­lysts to make mis­takes. After all, they are human. This is why we need a for­mal process of dou­ble check­ing to make sure that the data is free of error and there is ver­i­fi­ca­tion that all of the cor­rect, proper and val­i­dated steps were fol­lowed as artic­u­lated in the instruc­tions of the assay. The old carpenter’s adage comes to mind:

Mea­sure twice, cut once

But there is also another level of impor­tant dou­ble check­ing in the process referred to as error report­ing. I find it highly sus­pect (as you should) if a lab­o­ra­tory has  a for­mal­ized method of error report­ing, yet reports there are no errors. The fail­ure of Qual­ity Con­trol ini­tia­tives or actual test­ing unknowns is bound to hap­pen. To err is human, or so they say… To doc­u­ment this error is the sci­en­tif­i­cally and eth­i­cally cor­rect deci­sion. Ignor­ing it or cov­er­ing it up is not. It is not con­sid­ered a lab­o­ra­tory fail­ure unless it is reported out­side of the lab­o­ra­tory. Too often in foren­sic test­ing as per­formed in the United States today, the Qual­ity Assur­ance process is lit­tle more than a rub­ber stamp that equates any result gen­er­ated with a “good and valid” result. A good QA offi­cer should be more harsh than any edu­cated crim­i­nal defense attor­ney in exam­in­ing the data. Sadly, this is not the case.