Thursday, May 6, 2010

CPSIA - Numbers Don't Lie

One of my greatest frustrations with the CPSIA debate is its highly emotional tenor. It seems to me that the debate trades more in passion than in analysis. Since the big issue is safety, namely incidents of injury and death among children from consumer products, the resolution should depend on statistics and science - but the discussion NEVER goes there. Solutions based on a pure heart and good intentions won't solve a numbers problem. Many people involved in this law have little background in science. It is also clear that some people are playing fast and loose with data. That's a problem.

Many people use data relating to lead and childhood injuries that play on basic human emotions. After all, who would want another child to be injured, much less killed? That truism is justification for the CPSIA as "necessity", notwithstanding possible "unintended consequences". Even the mere mention of cost and effectiveness discredits opponents of the law.

A convenient example of this phenomenon is Henry Waxman's opening statement at last week's hearing. He called out three separate incidents of injury to kids: "Just over four years ago, a little boy named Jarnell Brown died of lead poisoning when he swallowed a metal charm that came with a pair of kids’ shoes. A year later, in 2007, two children became comatose and had to be hospitalized because of a children’s toy that turned out to have a toxic drug in it. That same year, millions of cribs were recalled for a simple defect that had caused multiple infant deaths."

These three groupings of injuries are unrelated and derive from radically different hazards. Jarnell Brown swallowed a jewelry charm made of lead. The two kids were affected by AquaDots, an unprecedented, one-of-a-kind chemical toy defect. Crib injuries are obviously of a different nature and involve a separately regulated category of child (infants) subject to unique risks. Despite the wildly disparate data, he concludes: "The bottom line was clear: our system for keeping children safe was broken."

While I think that the Cubs are pretty mediocre and so are the Bulls, I don't think Chicago is broken. This is apples-and-oranges. So goes the CPSIA debate.

Let me be clear, I am not saying Waxman INTENDED to mislead with his remarks - no, in fact this is how most people have treated data in support of the law. Few if anyone uses actual balanced injury or hazard data. Even the fear mongers' standard of "there's no safe level for lead" is almost obviously untrue but it strikes an emotional chord and is, at one level, hard to refute. Clearly, sprinkling lead on your bowl of cereal in the morning isn't recommended. That said, it is equally clear that lead is very infrequently deadly and since we all consume lead daily by breathing, eating and drinking, it must be tolerable (at a minimum) at certain dosage levels. Thus there has to be a "safe" level for lead - or else we'd all be as dumb as a box of rocks. I consider to be a simple observation.

So what's the truth? Data on recalls and the hazards uncovered by the CPSC is frankly hard to come by. I went through the exercise of preparing a 25-month analysis of lead-in-paint back in February 2009 and published my data. The data were great but frankly, I needed more. So I set out to prepare a much broader survey of recall data so we could really talk numbers. I have attached a data analysis of the 899 CPSC recalls of Children's Products between March 5, 1999 and April 15, 2010. This is slightly over 11 years of data and while it is a somewhat arbitrary period, I consider it statistically significant and useful to reason from.

A few caveats: I prepared this data with the assistance of several people and spent quite a bit of time cleaning it up. It is good data, but almost certainly inaccurate in some respect or respects. This is not intentional and I would certainly welcome notice of any errors you may find. The source of the data is posted recall notices on the CPSC website. The categorization of products and hazards might be considered quirky and could be misleading if you do not look at the details. However, overall, this is the best data I know of for an analysis of injuries and deaths from Children's Products.

The rationale behind this analysis is that official CPSC recall data over an 11-year period reflects the policies, judgment and decisions of the agency and is revealing about what the agency thought was and was not important during that period of time.

The spreadsheet has three worksheets. You have complete access to ALL of my data. I have incorporated hyperlinks to EVERY recall in the spreadsheet, so you can dig deeper if you are so inclined. The first sheet, entitled "Total by Category" is sorted by Product Category. The second sheet, entitled "Total by Year" sorts the same data by Recall Date. The final sheet, entitled "Summary by Product" includes FOUR ANALYSES pulled from the first two sheets. There is recall statistical data by (i) product category by hazard, (ii) hazard by product category, (iii) year by hazard, and (iv) hazard by year. Four slices of the data, never before seen.

Some interesting facts:
  • Grand Total Recalls: 899
  • Grand Total Children's Products Recalled: 3,128
  • Grand Total Units Recalled: 308,697,297 (remember, this is over 11 years)
  • Grand Total Injuries to Kids from Recalled Products: 2,381
  • Grand Total Deaths from Recalled Products: 35
Okay, let's unpack this data. First of all, this is inclusive of ALL Children's Products. That would include the recall of 150 million units of vending machine jewelry in 2004. It also includes all crib, bassinet, play pen and stroller recalls. Even common toy box recalls are included. You are welcome to react emotionally to these gross numbers, but please remember, this is a mishmash of 11 years of data at the macro level. We need to probe more deeply for real insights.

For perspective on these data, please consider CDC data on childhood mortality: the current mortality rate for kids age 1-4 is 4,631 per annum (leading causes of death: unintentional injuries and congenital malformations) and for kids age 5-14 is 6,149 per annum (leading causes of death: unintentional injuries and cancer). I believe unintentional injuries are principally car accidents. Anyhow, this implies an 11-year mortality rate of 118,580. Notably the annual mortality rates exclude kids 0-1. I don't have those numbers and haven't bothered to look for them. You get the picture. The CPSC has accounted for 35 deaths in the same period.

It bears noting that these statistics suggest that our markets are in fact rather safe. Injuries to children and deaths associated with Children's Products, while unacceptably high, are just 0.03% of the overall mortality statistics. We can and should continue to focus on improvements, but we should also have some perspective on our challenge. Polishing the apple is different than crisis management.

To further clarify the injury/death problem from lead, let's look at the breakout of injuries and deaths by hazard. Unlike my February 2009 analysis, this data shows all hazards and all recalls in the 11-year period. Please note that some of these hazards are very closely associated with one or two product categories. Lead is closely associated with jewelry. Lead-in-paint is largely associated with toys but touches almost all product categories. Falling and entrapment is a largely infant category. Strangulation is mainly clothing (certainly of late, with all the drawstring recalls) and infant products. With this data, you can see what comes from where.

Injuries and deaths from Children's Products in this period break down as follows:

  • Brake Failure 0, 0
  • Burns 74, 0
  • Cadmium 0, 0
  • Choking 150, 3
  • Collision 2, 0
  • Falling/entrapment 1803, 17
  • Fire hazard 4, 0
  • Illness 0, 0
  • Impalement 0, 0
  • Laceration 284, 0
  • Lead 3, 1 [The only death from lead in 11 years is Jarnell Brown, mentioned above.]
  • Lead-in-paint 1, 0 [That's right, ONE INJURY in 11 years, no deaths.]
  • Magnets 3, 0
  • Strangulation 26, 7
  • Suffocation 29, 7
A quick question: if you were running the CPSC and had access to this data, what would you focus on? Interestingly, if you look at lead and lead-in-paint recalls in the early years of this data, you will see something that seems somehow quaint now, namely very few recalls and in small quantities for lead and lead-in-paint.
  • 1999: 0 recalls [partial year]
  • 2000: 0 recalls
  • 2001: 5 recalls
  • 2002: 3 recalls
  • 2003: 4 recalls
  • 2004: 6 recalls
  • 2005: 10 recalls
  • 2006: 16 recalls
  • 2007: 98 recalls
  • 2008: 65 recalls
  • 2009: 29 recalls
  • 2010: 11 recalls [partial year]
Was the agency napping . . . or was the CPSC behaving rationally? What were they doing when they could have been catching lead-in-paint violations? Check my analysis: they were focusing on choking, falling/entrapment and laceration, presumably because these hazards were causing BY FAR the most injuries and deaths. Lead and lead-in-paint produced almost no injuries or deaths for 11 years. It is also worth noting that in 2007, under withering political and populist pressure, the agency changed its recall policy on lead and lead-in-paint to strict liability (every violation was recalled). In the early part of the decade, many lead violations were dealt with privately by the agency, making statistical comparisons impossible (and giving the incorrect impression of laxity by the agency at that time).

The data is also useful to give perspective on the effectiveness of the CPSIA's application of regulatory and corporate resources to new safety activities. In conjunction with last week's hearing, the HTA filed an economic analysis prepared by an independent safety consultant that projects aggregate annual CPSIA testing costs of $5.6 billion. Personally, I think that number is very low for the all-in economic impact from this law, but for the sake of argument, let's use it as THE number for regulatory compliance. So we are all going to spend $5.6 billion ANNUALLY to comply with this law regulating lead and phthalates. Over an 11-year period, ignoring inflation, our costs will be $61.6 billion. This expenditure is presumably designed to reduce the incidents of lead poisoning and phthalates. We have accounted for four injuries and one death. That's what our $61.6 billion is aimed at.

Consider this: if we really want to keep kids safe, we should be somewhat indifferent to HOW they are injured - we just want to protect them from harm. Thus, a bump on the head, a broken arm and lead poisoning are all the same, equally bad. So what if we spent at the same rate on ALL injuries and death identified in CPSC recalls, at the projected CPSIA spend rate? Our cost would be $61.6 billion / 5 (11-year lead and lead-in-paint injuries and deaths) X 2416 (total injuries and deaths in the same period). Sitting down? The total compliance cost we would bear over 11 years would be $29.8 trillion ($2.7 trillion per year). The current national debt is $12.95 trillion. Please also consider that our annual safety expenditures would exceed the expected receipts of the federal government this year ($2.4 trillion). If the federal government only funded us, the government would still run a $300 billion deficit this year.

Call me a worrywart, but I think that kind of safety spending is a bit over the top.

The debate over safety has dramatic economic implications. Safety is a principal concern of the industries serving children's markets, of course, but if we are crippled economically we won't be around to make safe products. There is a sensible balance that can be achieved. The data suggests that the CPSC actually knows what it's doing (at least it did before it became a political football) and can properly allocate its resources.

If the CPSC were entitled to focus on real safety risks (based on data), not the phobias of politicians and consumer groups, and if they could be convinced to stop trying to run our businesses for us, I think we could reduce and refocus safety spending and become much more effective in improving safety.

Numbers don't lie. It's time to reassess and amend the CPSIA. And if the Dems won't let us do that, it's time to reassess them and Congress as a whole.


jennifer said...

amazing. i wish the politicians would unpack the data and analyze it. thank you!

Jennifer Taggart, TheSmartMama said...

Thank you for taking the time to compile this data.

That being said, it seems to me that as every government agency admits, lead in residential housing units remains the primary source of lead exposure for children - from peeling paint and lead contaminated dust. Yet, the programs to reduce lead in residential housing don't get much funding or attention. And the public's knowledge of the hazard is woefully inadequate or the simple steps to avoid exposure (wet wipe cleaning, washing hands, taking off shoes, HEPA vacuum, isolation of hazard areas, etc.). If just a fraction of the money used was spent on those programs, the hazard of lead would be dramatically reduced at a much better cost per harm avoided ratio. And if we got rid of lead ammunition, another source of lead would be gone at a much better cost per harm avoided. And if we screened or tested folk remedies and educated populations about the risk of lead in remedies like azarcon and greta, another source of lead would be gone at a much better cost per harm avoided.

Oh, but that would be too sensible, right?

Jen said...

Very interesting Rick. I would love to see the Freakonomics guys ask some interesting questions of the data with respect to legislative and policy decisions.

kathleen said...

It's not just inciting fear that is the problem (your protests to the unreasonableness of it duly noted). The issue is assigned responsibility however displaced (!).

Producers are an easy target, the bigger the better. If parents and interest groups have those to hold culpable, then everyone else (parents etc) is off the hook. Any other alternative amounts to lending credence to the likes of those Free-Range Kids communists.

So imo, we have two problems. One is unreasonable fear being used to manipulate the populace and the psychological desire of individuals to have a ready made scapegoat in the event things go awry so they don't need to make any efforts at modifying their behaviors or choices. You know, a get out of jail free card.

Everybody wants a target, so how do you deal with that? Numbers aren't going to persuade them, they're not operating at that level. Tragically enough, logic and reason have nothing to do with it.

halojones-fan said...

@Jennifer: The problem is that working on lead-paint reduction doesn't do ANYTHING to create a stealth tariff on toys manufactured in China...

J-momma said...

as a statistician, I can assure you that numbers can assert whatever deception one might wish. In the case of your analysis, however, I believe they speak the truth, with one meaningful exception. I have read the entire analysis from which you derive the 5.6 b annual cost. The author asserts that 5 million items would require an average of 5 tests. I believe this estimate to be extremely low by a factor of 5 to 10 at least. Consider for a moment just one aspect of "reasonable testing" - Verification of 3rd party results According to the NPR released April 1, we will need take reasonable care to verify EACH applicable children's product safety rule for which the product is tested. Now, the Commission does not define the frequency of this verification schema, but based on their previous requirements, I would imagine that "requiring a high degree of assurance" would require at least yearly cross verification for each of the 5 million items for each of the 5 tests being factored. So, at a minimum, I'd say it would be safe to double your figures.

Rick Woldenberg, Chairman - Learning Resources Inc. said...

Okay, truth serum time. I said I thought the $5.6 billion number was low but took it as the number for my analysis, basically on the grounds that $5.6 billion is a LOT of money. Close enough "for government work" to make my point. My honest opinion is that the actual cost to comply with the CSPIA is TRIPLE this number. Remember that there are a LOT of companies affected. Everybody is testing and retesting everything, and everybody is labeling and hiring QC staffs, filing needless paperwork, employing lawyers, flying to "two day meetings" and so on. It is a terrible, unproductive tax.

If you take $15 billion as "the number", the costs are even more staggering and shocking. I didn't want to use that in my original essay because it drove such astounding numbers that I felt I might lose credibility. Roughly speaking, at $15 billion in annual spend, the total industry cost (if applied equally to every cited injury to children) would be more than $8 trillion per annum. That's more than I have to spend . . . .

The U.S. 2010 GDP is expected to be $14.2 trillion this year. So what if more than half of our economic output is spent testing children's products?