Basic Math Errors Made by the CDC
Our Pre-Print on CDC Errors is Now Online
I was working on another Substack article, but things got a bit derailed today, when the pre-print that I co-authored with Dr. Vinay Prasad, Dr. Tracy Beth Hoeg, and Dr. Alyson Haslam was posted on the pre-print server. This is the project I mentioned I had been working on earlier. Now I can share the details!
If you’ve been following me for a while, you’ll know about some of the errors covered in this paper. Several are related to the completely inaccurate Data Tracker demographics, some are related to the misuse of a flawed pre-print to claim Covid as a top 5 cause of death in children, and others are issues with pediatric hospitalization data, variant percentages, and other issues that I’ve tweeted about before. If you want to see all the details, I encourage you to download the paper and go through the list yourself. Each of the errors include links to sources so you can look into them further if you’d like. I’ll probably write another Substack article where I dig in deeper about some of the errors, so if there are any you’d like to hear about more, let me know!
Are these errors, or are they intentional?
I’ve already seen a lot of feedback from people who feel strongly that these aren’t errors, but outright lies. There are several ways that data errors like these occur:
Simple mistakes (typos happen)
Mistakes caused by lack of expertise or effort (these aren’t accidents, but “acts of negligence” as my dad would say)
Errors caused by systematic bias (mistakes aren’t identified because the data support their priors)
Intentional misrepresentations (noble lies)
While it is hard to know with certainty the reason behind some of these false statements, the fact that so many of the errors exaggerate the risk, particularly to children, raises concerns about real or perceived systematic bias. We discuss this in our paper:
“If errors fall in one direction preferentially – either overstating the risk of the virus or understating it – trust may be eroded, and the potential for bias within the agency is raised.”
During the pandemic, the CDC was tasked with reporting and analyzing Covid data, as well as setting policy recommendations that in many cases were codified into local/state/federal mandates. As a result, it is particularly important for the data to be without error, and people must trust the veracity of the data being presented.
“These errors have been made repeatedly and were likely to have affected discussion of pandemic policies. During the years the errors occurred, CDC’s guidance repeatedly called for restrictions being placed on children, including school closures, mask mandates, and strong recommendations for vaccinations and multiple boosters even among children who have recovered from the virus.”
Even a perceived bias within the CDC affects the public’s trust of both the data and the resulting policies. It leads reasonable people to wonder if the policies are data-driven, or if the data is policy-driven.
What next?
More work needs to be done to investigate the accuracy of the data and analysis done by CDC during Covid. This paper only addresses the most basic mathematical and statistical errors that we compiled. The errors in the paper are matters of objective fact, not subjective claims that are a matter of interpretation.
“Our investigation, essentially a fact checking exercise, suggests 1) a greater diligence is needed to avoid errors in public health data, and 2) that the federal entity responsible for reporting health statistics should be firewalled from the entity setting policy, for concerns of real or perceived systematic bias in errors.”
The CDC also has also been widely criticized for many poorly-designed or misrepresented studies and other analyses. Even many advocates of CDC policies on masking and vaccines have been critical of the analysis and conclusions of some of their MMWRs, which are always in support of official CDC positions. In addition, those looking at the data often have significant disagreements about the best policy recommendations. But underlying all of these debates is an understanding that the CDC must provide accurate information, and when errors are identified, they should be corrected promptly and not repeated. In many instances, the CDC has failed to meet that basic standard.
Congratulations Kelley, you are an inspiration!
Fantastic work! Congrats first author! It is so important to have fact checkers like you.