By Radley Balko May 5 at 3:24 PM

THE WATCH

My favorite story of forensics malfeasance emerged from the North Carolina state crime lab a few years ago. I wrote about it at the time:

The relationships between SBI crime lab researchers and North Carolina prosecutors aren’t just cozy, they’re downright cuddly.The News & Observer reports that in one case two blood-spatter specialists ran through multiple experiments in order to produce even one that would make the blood patterns on a defendant’s shorts support the prosecution’s case. The two analysts are seen on video high-fiving after finally producing the desired result.

For those clinging to the notion that analysis in a law enforcement-managed laboratory can be independent, the newspaper uncovered prosecutor reviews of crime lab analysts indicating the contrary. In 2003, for example, prosecutor Ann Kirby, wrote in a review of a drug analyst, “If Lisa Edwards gets any better on the witness stand, the Johnston County defense bar is going to try and have her banned from the county!”

These weren’t a few rogue analysts; the crime lab’s problems extend across a wide array of forensic disciplines. Until 1997, the lab’s serology unit didn’t release negative test results as a matter of policy. If tests showed that a substance that police claimed was blood wasn’t in fact blood, analysts simply kept those results to themselves.

That story really hammers home the incentive problems in U.S. crime labs. Crime lab analysts are supposed to be neutral parties interested only in getting the science right. But the system is often structured in a way that makes them part of the prosecution’s “team.” In fact in many jurisdictions, crime labs actually get paid per conviction, not per analysis—about as clear a perversion of objectivity as one can imagine. Of course, the pressures and incentives needn’t be that explicit. For example, just knowing extraneous details about a case can produce cognitive bias, even in as accepted a field as DNA analysis.

There are two fundamental things that need to be done to reform the field of forensics. The first is to purge the courts of specialties that have no basis in science. With the fields that are left, we have to turn these incentives around, so that the performance of crime lab technicians is measured only on whether or not they perform accurate analyses.

Unfortunately, there has been very little discussion of the incentive problem among the various federal agencies charged with looking into reform. Possibly moving crime labs out from under offices of state attorney general or state police organizations is about as far as the suggestions go. They need to go further. Over at Reason,Roger Koppl, a professor of finance in Syracuse University’s Whitman School of Management and a faculty fellow in the school’s Forensic and National Security Sciences Institute, has some concrete suggestions.

• Cross-lab redundancy.A jurisdiction should contain several competing forensic labs. Some evidence should be chosen at random for multiple testing at other labs. This creates checks and balances.
• Independence.Put crime labs under the department of health, not the cops.
• Statistical review.Compare the results of different labs and look for statistical anomalies. An investigation may reveal bad practices to be eliminated or good practices to be emulated.
• Sequential unmasking.Forensic scientists can be biased by scientifically irrelevant information such as the criminal history of the suspect. Sequential unmasking is an administrative control process similar to that used in double-blind research studies. It prevents forensic scientists from learning potentially biasing information until after they have made their determinations.
• Forensic counsel for the indigent. Most criminal defendants cannot afford their own forensic experts. Basic fairness says that they should have a right to their own experts just like they have a right to counsel. A voucher system is the best way to provide defense experts.

The cross-lab redundancy is key. If an analyst knows that every third or fourth test he performs — but not which test — will be repeated by another analyst in another lab, you’ve dramatically shifted the incentive. Even if he reports directly to the prosecutor, that analyst’s prime objective is to get the test right. If he doesn’t, sooner or later he’s going to get caught, embarrassed and possibly out of a job. Without this reform, the incentives are dramatically different.

By the way, this isn’t to suggest that crime lab analysts are corrupt. Cognitive bias creep into the work of even the most conscientious analysts. Scientists outside of forensics understand this, and take precautions to guard against it, like double-blind testing and peer review. Forensic analysis isn’t quite the same as the scientific method, mostly because it has different objectives. But if we’re going to give it the weight of science, we need to find away to subject it to basic scientific principals.

These ideas aren’t new. In fact, Koppl and I suggested them in Slate back in 2008. Back then, it was difficult to get policymakers even to acknowledge that there was a problem. That’s no longer the case. There is now a broad consensus that we’ve been using flawed science in our courtrooms, and that this has resulted in the conviction of innocent people (which of course often means guilty people go free). But the fact that a thinker like Koppl isn’t sitting on the president’s Forensic Science Commission while, for example, the federal subcommittee in charge of investigating bite mark evidence is loaded with practitioners of the very field that the committee should be investigating, suggests that the moment for forensics reform is in danger of passing us by, and that the apparatus put in place to do it may have already been captured.

Radley Balko blogs about criminal justice, the drug war and civil liberties for The Washington Post. He is the author of the book “Rise of the Warrior Cop: The Militarization of America’s Police Forces.”