Always Check the Denominator
No, the risk of critical Covid disease is not 4% among healthy children.
I’ve been very busy lately with my actual job and some other things, so I haven’t been writing much lately. I paused paid subscriptions for the summer to try and get caught up. I’m not taking the summer off - I plan to start writing again even while payments are paused. Hopefully you’ll be hearing more from me soon! Today seemed like a good day to start…
I woke up to this outrageous claim from CIDRAP in my Twitter feed:
This is not the first time that I’ve written about CIDRAP amplifying misleading claims from a Covid study to amp up fears about Covid and kids. They also misreported a study about Long Covid, claiming that 84% of Covid survivors had Long Covid symptoms 2 years later. Like that ridiculous Long Covid claim, the claim from their current tweet and article on Covid risks among kids just doesn’t pass a basic sanity check. Michael Mina did some basic math to show how absurd this claim is:
Unfortunately, the reporters at CIDRAP (along with many other Covid journalists and even some scientists) seem to lack the common sense to do these kinds of sanity checks on the data — because this keeps happening.
A Meta-Analysis of “Risk Factors for Pediatric Critical COVID-19”
The study, “Risk Factors for Pediatric Critical COVID-19: A Systematic Review and Meta-Analysis” is primarily about which comorbidities are risk factors for severe Covid — it’s focused on severe cases and kids with comorbidities. However, the study does include the 4% claim cited by CIDRAP. I can’t access the full text of the published paper from the Journal of the Pediatric Infectious Diseases Society, but the study’s abstract does state: “In previously healthy children, the absolute risk of critical disease from COVID-19 was 4% (95% CI, 1%-10%).” I wanted more details of the underlying data, so I found the pre-print of the study, and the following section of the Results is where this 4% statistic comes from:
This small section is the only mention of outcomes in children without comorbidities, and it doesn’t include nearly enough caveats about the underlying data. It is a poor conclusion to draw and shouldn’t have been included in the abstract at all in my opinion. The abstract even says that “the absolute risk for critical COVID-19 in children and adolescents without underlying health conditions is relatively low,” which doesn’t seem consistent with a 4% “absolute risk of critical disease” from their results. In reality, the absolute risk among previously healthy children is at least an order of magnitude lower than that.
Underlying Study Data
It should be obvious to the casual observer that the 4% was calculated from a population that doesn’t represent typical healthy children. So I looked up each of the 6 studies included in this subset of studies from the meta-analysis. Of the 6 studies that looked at medical complexity, 4 were among hospitalized children (3 of the 4 in Brazil)1, one was among 12-17-year-olds with symptomatic Covid recorded in medical records2, and one was a large study of children with documented Covid in Mexico3.
Also, all of the studies were from early in the pandemic (2020 and part of 2021), so they were primarily first infections, they were primarily pre-vaccine, and all were pre-Omicron. So they don’t tell us much about the current situation, even among children who may be hospitalized with Covid in 2024.
The large study in Mexico included over 130,000 children and found 0.3% of children with documented Covid infections and no comorbidities died between March 2020 and mid-June 2021. And that is still biased to more severe cases, because many asymptomatic and mild infections aren’t documented in medical records. Despite the overwhelming findings from this large study of children from the broader community (not just hospitalized children), the study authors calculated an absolute risk from a small subset of children from these 6 studies (less than 16,000 patients). They don’t provide any further details the subset that they used for their calculations, but clearly the studies of hospitalized children and symptomatic teens skewed the results significantly.
Denominators Matter
There’s been much discussion about the importance of denominators throughout the pandemic, but scientists, journalists, and others keep making the same mistakes. When we use percentages to talk about Covid outcomes or other things, we’re talking about a fraction of the population. But it’s super important to understand WHICH population we’re talking about. Outcomes among hospitalized patients, or among the elderly, differ greatly from outcomes in the general population.
Just like the percentage of adults who ride mountain bikes is much higher among attendees at a mountain biking race than among all adults, the percentage of children with critical Covid is obviously much higher among children who are hospitalized for Covid than among all children. We see this same error with Long Covid studies, where the findings are percentages among patients at a Long Covid clinic, or among patients who already have Long Covid, but are often reported as if they are percentages among a representative sample of the population.
The percentage of children with critical Covid outcomes is obviously much higher among children who are hospitalized for Covid than among all children.
The Sad State of Covid Science and Journalism
The authors of this study were irresponsible to include the claim that “the absolute risk of critical disease from COVID-19 was 4%” among healthy children without comorbidities, without explaining that their data was heavily biased toward patients already hospitalized with COVID. Without that caveat, the 4% claim is a gross misrepresentation of the underlying data. And this study isn’t from some obscure scientist at a no-name institution — the senior author, Dr. Carlos Oliveira, is an Assistant Professor of Pediatrics, Biostatistics, and Biomedical Informatics & Data Science at Yale! Surely he knows that 4% of American children didn’t have severe Covid outcomes. Why include that misleading statistic in the paper?!?
I also think the writers at CIDRAP needs to do a better job of understanding and vetting the studies they report on to make sure their reporting is accurate. They are a trusted voice for many people, and their studies often get amplified by the Zero Covid crowd on Twitter. It is profoundly unhelpful to continue scaring people with these misleading and unrealistic statistics. Journalists need to understand the harm they cause by publishing these irresponsible claims.
By the way… I’m already seeing this same error with H5N1 (bird flu) in the media as well, and will probably write more on that later. Many news reports refer to a 50% “fatality rate”, but that’s only among known cases that were previously documented. Among the 3 cases identified so far in the US (one in 2022 and two this year), none have died. The two known cases this year have only experienced eye infections.
UPDATE: The moment I published this, I saw that another case of H5N1 has been identified in the US. That patient did have respiratory symptoms but is recovering. (5/30/24)
(A) Ward JL, Harwood R, Smith C, Kenny S, Clark M, Davis PJ, et al. Risk factors for PICU admission and death among children and young people hospitalized with COVID-19 and PIMS-TS in England during the first pandemic year. Nat Med. 2022 Jan;28(1):193–200.
(B) Oliveira EA, Colosimo EA, Simoes ESAC, Mak RH, Martelli DB, Silva LR, et al. Clinical characteristics and risk factors for death among hospitalised children and adolescents with COVID-19 in Brazil: an analysis of a nationwide database. Lancet Child Adolesc Health. 2021 Aug;5(8):559–68.
(C) Hendler JV, Miranda do Lago P, Muller GC, Santana JC, Piva JP, Daudt LE. Risk factors for severe COVID-19 infection in Brazilian children. Braz J Infect Dis. 2021 Nov-Dec;25(6):101650
(D) Horta M, Ribeiro GJC, Campos NOB, de Oliveira DR, de Almeida Carvalho LM, de Castro Zocrato K, et al. ICU Admission, Invasive Mechanical Ventilation, and Mortality among Children and Adolescents Hospitalized for COVID-19 in a Private Healthcare System. Int J Pediatr. 2023;2023:1698407.
They just keep cranking out the propaganda. Humanity disappoints me in how many believe this garbage.
As with all modern journalism "if it bleeds, it leads!'