CPSIA - Why Does The Press Push Bad Ideas?

Last week, Justin Pritchard of the Associated Press again pressed the point that glassware with lead in the enamel should be recalled. He expressed shock at the "stunning" news that the CPSC reconsidered its view that the glasses are subject to the noxious lead regulations of the CPSIA.

Why did Mr. Pritchard find this outcome "stunning"? After all, in response to a question about the risks involved in the glasses, Pritchard had this to say in a TV interview: "The [lead and cadmium] levels are low in the sense of . . . no one is going to touch this glass, put their hand to their mouth and fall ill. This is a low level over time concern." If the glasses are a low risk, why would he expect a federal regulator to waste time or resources on them?

Even more remarkably, Pritchard knows that the McDonalds Shrek glasses were found to be non-toxic by the CPSC. He broke the McDonalds story. The Shrek glasses present precisely the same "issue". He also knows lead is only restricted in children's products and that enamel coatings containing lead are permitted explicitly in the law (16 CFR 1303.2(b)(1)). There is no evidence that the presence of lead in the enamel has ever injured anyone. Ever.

So why is Pritchard continuing to push a story that he knows is defective? This puts it kindly. Let's rule out that he is seeking a Pulitzer or has an ill-motive. Why would he do this?

Of course, we know there is a bias in reporting and in investigating that favors reporting "bad news". Good news is not really considered news at all (except on the sports page). The media's incentive is to publish terrifying stories - it sells papers and banner ads, and it's natural for Congress to push legislation to save us from poorly understood threats as an extension of this trend. But something else is at play, it turns out.

This subject is analyzed in an interesting article by Jonah Lehrer in this week's New Yorker magazine entitled "The Truth Wears Off". Lehrer tries to explain why replication of scientific studies tends to show declining results over time. This is quite unexpected given that scientific studies are subject to peer reviews and are often published by periodicals with their own high standards of review. Lehrer notes that in small studies, weird results can show up (such as a 1930's study which claimed that one Duke University student had ESP but later retesting revealed the student's rapidly diminishing extrasensory powers . . .). In larger pools of data, results revert to a mean (this is called "funneling"). However even statistical significance doesn't explain the phenomenon. Lehrer shows that we only get to see certain slices of data. Most data won't be published because it's not interesting or doesn't confirm prejudices.

Put into a CPSIA context, Lehrer implicitly argues that media won't write a story announcing that lead-in-enamel on your glassware is safe. Nor that you were always fine and your children weren't in danger. Nor that there have been few injuries from lead in any children's products. Nor that the few known injuries in the context of the large volume of products in use is actually a GOOD result. Nor that there are no identified victims of "phthalate poisoning" or that incidents of cadmium poisoning in American children are virtually unknown. The excuse - it's not "newsworthy". What's the reality?

The reality is that we are exposed to a very imbalanced set of data. Quoting Michael Jennions, a biologist at the Australian National University, Lehrer argues that "the tendency of scientists and scientific journals [is] to prefer positive data over null results, which is what happens when no effect is found." If the null set (the "everything's fine" news) doesn't get reported, what does? Says Richard Palmer, a biologist at the University of Alberta, "We cannot escape the troubling conclusion that some - perhaps many - cherished generalities are at best exaggerated in their biological significance and at worst a collective illusion nurtured by strong a-priori beliefs often repeated."

The same mantra over and over? The words "Rachel Weintraub" suddenly pop into my mind.

Lehrer continues: "[T]he problem seems to be one of subtle omissions and unconscious misperceptions, as researchers struggle to make sense of their results. Stephen Jay Gould referred to this as the 'shoehorning' process." Referring to studies in Asia that consistently confirm that acupuncture is effective, and studies in the West that show much poorer results, "Palmer notes, this wide discrepancy suggests that scientists find ways to confirm their preferred hypothesis, disregarding what they don't want to see. Our beliefs are a form of blindness."

Or to quote Robert Adler, anecdotes aren't evidence.

John Ioannidis, an epidemiologist at Stanford University who once published a study entitled "Why Most Published Research Findings are False", calls the phenomenon "significance chasing" where scientists play with numbers trying to find "anything that seems worthy". In a news context, this is the same as Pritchard fingering the Super Hero glasses on the grounds that there is lead in the enamel even though he knows the Shrek glasses were safe. Maybe these other glasses are a problem?! Jeff Plungis of Bloomberg published an article on lead in Christmas light wires on December 8th because he apparently thought it was "interesting" and not well-known. Same thing.

Ioannidis says "It feels good to validate a hypothesis. It feels even better when you've got a financial interest in the idea or your career depends on it. And that's why even after a claim has been systematically disproved . . .you still see some stubborn researchers citing the first few studies that show a strong effect. They really want to believe that it's true." [Emphasis added]

Lehrer's article is a great read, I recommend it to you.

So you can stop scratching your head. Pritchard and Plungis, Adler and Tenenbaum, Waxman and Schakowsky, Weintraub and Green, will all continue to beat the same drum. They know they're right . . . they just can't prove it. And they will continue to repeat themselves in spite of the facts of this case:

• There are (virtually) no known victims.
• The impact of the law cannot be measured.
• The nexus between lead in children's products and purported injury to children is not proven. This means that the inclusion in the law of so many formerly unregulated categories of goods is absolutely unjustified.
• The benefits of prophylactic testing has been disproved by the passage of time - the last 29 months.
• The law targets small business and lets big business off the hook. Even since passage of the CPSIA, it is clear from data that big business are responsible for headline recalls.

I guess the media keeps on publishing these stories because it's human nature. Unfortunately, many jobs and many futures have been damaged in the service of a human weakness. I like to think we can rise about such limitations. It is in the hands of the CPSC and Congress to solve this problem.

Let's hope they do their job . . . sometime really soon.