Editor's note: The following is adapted from an interview appearing in the Italian magazine Va' Pensiero.
Suspicious Numbers
An interview with Peter Doshi,
graduate student, Massachusetts Institute
of Technology
Va' Pensiero n° 285 (Italy) [original in Italian]
20 December 2006
Va' Pensiero (VP): In British Medical Journal (Dec 10, 2005), you wrote that "US data on influenza deaths are a mess." Can you explain your concern?
Peter Doshi (PD): There are many reasons why the data is a mess. Take, for example, the CDC's regularly quoted statistic about the influenza: that it kills an average of 36,000 Americans each year. This number is not a body count, but is instead an estimate produced by a statistical model of questionable validity. When the figures this model produced are compared against officially recorded influenza deaths, one finds that while the CDC's statistical modeling shows a sharp 80% rise in influenza-associated deaths over the 1990s versus the 1980s, the recorded influenza deaths show a 30% decrease over the same time period.
This comparison reveals another problem. Most people are not aware that the CDC maintains two types of data on influenza deaths. One is their statistical-model-produced estimates (also known as "influenza-associated deaths"); the other is recorded deaths. While the public is frequently reminded about their statistical projections--e.g. the 36,000 figure--the problem is that the public is never told about the recorded deaths. If one looks at recorded influenza deaths over the 1990s, one will find an average of 1197 deaths per year--or just about one thirtieth of the number of influenza-associated deaths that they are advertising so heavily.
Now, this is not to say that the category of influenza-associated death is meaningless. The CDC argues that their recorded flu death figures are underestimating the true toll of influenza, and so they employ these statistical methods to produce what they feel are more accurate estimates of the impact of influenza. But the problem here is that there seems to be no relationship between their estimates and their recorded numbers. As I pointed out above, while their estimates are showing sharp increases, the recorded influenza death data shows a decrease in deaths over the same time period.
Another reason why the data is a mess is because there is so little hard data to work with. Of the 257 total deaths officially classified as influenza-caused in 2001, in only 18 cases was the influenza virus actually confirmed to be present. Many large statistical projections and much money is being spent on a very limited understanding of the true impact of influenza.
VP: The figures reported by the Centers for Disease Control seem to be a misrepresentation--but are they consistent with data gathered from other national agencies?
PD: Good question, complicated answer. In short, it depends on what part of the results one looks at.
Take for example the results of Lone Simonsen and colleagues from the National Institutes of Health (Am. J. Epidemiol. 2006;163(2):181-187), prominent influenza researchers who like myself have previously criticized the CDC's model. Their statistical modeling results are consistent with CDC estimates on some levels and inconsistent on other levels.
The results are consistent, for example, in that they take for granted the same questionable assumptions of a privileged relationship between influenza and pneumonia to derive their statistics. And they are consistent in that they assume that influenza plays a significant role in elevating so-called "excess deaths" (a jump above the expected mortality rates) in the winter--an assumption challenged by other influenza researchers who claim cold weather stress may be the underlying cause of the "excess deaths".
Beyond questionable assumptions, if we were to only look at the results of these statistical models, we would get an equally confused picture. Like the CDC, the NIH group published statistics of influenza-associated death in three categories: (1) influenza-associated underlying pneumonia and influenza deaths; (2) influenza-associated underlying respiratory and circulatory deaths; (3) influenza-associated all-cause deaths. Thus the CDC and NIH statistical models produce not one, but three influenza-associated death figures. Of these three statistics, somebody must be deciding which figure to present to the public, as most people are unaware that the 36,000 figure represents just one of the three estimates computed by the CDC's statistical model.
If one compares the NIH and CDC's results and only looks at is the second category, the figures are similar. But if one looks at the first category (influenza-associated underlying pneumonia and influenza death), the CDC's statistical estimate is entirely inconsistent with the NIH estimate: the CDC claims around 6,000 deaths/year between 1976-1999 and the NIH claims over 14,000 deaths/year between 1979-2001--a huge gap.
But one must be careful here. It is not sufficient to judge the quality of statistical estimates by their similarity in results. One must keep in mind the assumptions built into the models, and always be looking for ways to validate these models with other empirical observations.
VP: A book has recently been published in Italy by Wolfgang Sofsky. "Governments," he says, "are always looking for reasons to elevate concern among citizens, in order to easily control their dissatisfaction." False alarms, he says, are actually powerful weapons in the hands of Governments. What is your opinion about this?
PD: Using fear to motivate people to action is one of the oldest tricks in the book, and I think is the basic framework one needs to understand the majority of U.S. foreign policy since 9/11. Look at the so-called threat of anthrax. Anthrax is an exceedingly rare infection, but you wouldn't know that judging from how much concern there is about it. I think the statistics will show that, including the anthrax attacks of 2001, more Americans die from lightning in an average year than have died from anthrax in the last 100 years. But make no mistake: the anthrax attacks of 2001 were exceptionally successful agents of psychological warfare. They fuelled fear, and the attacks were used to justify increases in anti-bioterrorism budgets. Some have said that the anthrax attacks were a key factor in convincing Congress to pass the controversial USA Patriot Act as well. Thus political mountains can be made of epidemiological molehills.
VP: The CDC (and other public agencies worldwide) is working in the interest of vaccine manufacturers, generating more demand for their products. Do you think that this is an unavoidable occurrence of converging interests?
PD: Over the past few years, serious conflicts of interest have been revealed in connection with the National Institutes of Health (NIH), the Institute of Medicine (IOM), and the Food and Drug Administration (FDA). With the CDC, it is true that their marketing of influenza vaccine to the public and doctors alike works in the vaccine manufacturers' interest. Whether these are vested interests or converging interests, I do not know.
What I do know is that preceding the "National Influenza Vaccination Summit" of 2004, influenza vaccine demand had been low early into the 2003 flu season. Glen Nowak of the National Immunization Program (part of the CDC) went on National Public Radio and recalled: "At that point, the manufacturers were telling us that they weren't receiving a lot of orders for vaccine for use in November or even December. It really did look like we needed to do something to encourage people to get a flu shot."
I do not think this situation is an unavoidable occurrence of converging interests. If influenza truly poses a grave threat to society and if current influenza vaccines were a truly safe and efficacious way of beating the influenza, why can't the free market take care of the rest? Why would it need special marketing by the government? Major sellers like paracetamol, aspirin, and Viagra seem to be selling fine without government marketing blitzes.
Posted: March 19, 2007
Scienza e Democrazia/Science and Democracy