Another Misleading Long Covid Study

What's behind these scary rates of Long Covid symptoms?

Taylor Lorenz shared an article about a recent Long Covid study out of Northwestern, and it has been widely retweeted among Zero Covid Twitter. As with most Long Covid studies and accompanying articles, it’s a masterclass in lying with statistics. It’s clear that the study authors are motivated to exaggerate Long Covid. The senior authors, neurologist Dr. Igor Koralnik and pulmonologist Dr. Marc Sala, are the co-directors of Northwestern Medicine Comprehensive COVID Center, so they have a vested interest in growing the business of Long Covid care.

“Alarming Impact of Long Covid”

The study analyzed data from 1,802 patients, and these are some of the key rates of symptoms being reported from the study in the news:

• 85% reported decreased quality of life

• 51% had cognitive impairment

• 45% had altered lung function

• 83% had abnormal CT chest scans

• 12% had elevated heart rate on rhythm monitoring

At first glance, that sounds bad, but let’s look a little closer… This study was done based on data from the Northwestern Medicine Comprehensive COVID Center, a multidisciplinary Long Covid specialty center. These are not a random selection of typical Long Covid patients, but patients who sought treatment for Long Covid from a specialized center, so they are more likely to have complex or severe Long Covid symptoms. Furthermore, the center began in the neurology department at Northwestern and may have attracted more patients with neurological issues.

On top of that, the percentages above are not out of all 1,802 patients seen by the center. They are out of the subset of patients who received testing for each particular issue. It’s more informative when you see the raw numbers for each of those percentages below, as stated in the paper’s results section:

• 742/878 (85%) reported decreased quality of life

• 284/553 (51%) had cognitive impairment

• 195/434 (44.9%) had alteration of lung function

• 249/299 (83.3%) had abnormal CT chest scans

• 14/116 (12.1%) had elevated heart rate on rhythm monitoring

The FOX32 article at least says that the percentages are “among those tested” (though doesn’t quantify that), but other articles leave that caveat out, making them not just misleading, but factually incorrect. The news articles seem to be the result of a recent press release put out by Northwestern Medicine about the study and their Comprehensive COVID-19 Center.

Analysis of Selected Findings

Abnormal CT Chest Scans

The most troubling percentage from that list — 83% with abnormal CT chest scans — makes more sense when you break down who was tested. Less than a third of the patients in the study visited the pulmonology clinic, and only 299 (16.6%) patients were given chest CTs based on their symptoms. Of those patients, 83% (249/299) had abnormal findings on their CT scans. Among those patients, there were notable differences between hospitalized and non-hospitalized patients. The data in Table 3 shows that 93% (138/149) of hospitalized patients and 74% (111/150) of non-hospitalized patients had abnormal findings. When you look at the total subgroup of non-hospitalized patients in the study, only 7.6% (111/1452) of non-hospitalized patients at this Long Covid specialty center had abnormal chest CTs.

And to make matters worse… I’m not certain that the 111 number is even correct, based on more detailed data in Table 3 showing data on the specific abnormal findings. If you add up the positive findings for non-hospitalized patients in the table (the numerators inside the red box), there are only 90 instances of abnormal CT findings, which would indicate there were no more than 90 non-hospitalized patients with abnormal findings. It seems like either the number of people listed with “none of the above” CT findings or the denominators shown are too low. Either error would reduce the percentage of non-hospitalized patients with abnormal CT findings.

Reduced Quality of Life

I also found a potential error with the reported 85% of those who reported lower quality of life via “questionnaires for self-perceived quality of life in domains of cognitive function…, fatigue, sleep disturbance, anxiety, and depression”. The data in Table 2 shows that number to be 742/916 (81%), but the results section says 742/878 (85%). The number 878 doesn’t show up anywhere else in the paper.

Regardless of whether the number is 81% or 85%, it’s not surprising that most people who met the following criteria for an appointment with the neurology clinic reported quality of life issues:

“Clinical criteria for appointment to the neurology clinic included individuals with confirmed diagnosis of SARS-CoV-2 infection or carrying a diagnosis of COVID-19 presenting with brain fog, cognitive problems, confusion, headache, decreased sense of smell and taste, weakness, numbness, muscle pain and loss of consciousness. People with no diagnosis of COVID-19 but worried they have been exposed to SARS-CoV-2 or having neurologic symptoms listed above were evaluated as well.”

Elevated Heart Rate

Despite a lot of talk about cardiovascular issues among Long Covid patients, that is the specialty that had the least number of clinic appointments in the study, and the area that had the lowest rates of identified issues. The finding they noted in their results was that 12.1% had an elevated heart rate (average ≥90 bpm) — of the 116 patients who underwent rhythm monitoring, which is a relatively common test for anyone experiencing heart rhythm issues. (I’ve used a rhythm monitor myself twice prior to Covid.)

Among the 116 rhythm monitors performed, only “2 (1.7%) showed clinically significant ventricular arrhythmias” and only one identified “new atrial fibrillation.” Somehow that reassuring information didn’t make it into the summaries of the study. Weird, huh?

Incidence of Long Covid

Although this Northwestern study wasn’t about the incidence rate of Long Covid, the paper did make two separate and very different claims about Long Covid incidence:

• “Long COVID affects 10%-30%², ⁵¹, ⁵² of SARS-CoV-2-infected individuals, depending on variant and severity of acute illness.”

• “The incidence of long COVID remains uncertain, with estimates ranging from 30% to 50% among COVID-19 survivors.^10”

The study authors couldn’t decide whether to claim Long Covid affects 10-30% or 30-50% of people who have Covid-19?

I couldn’t find any obvious estimates of Long Covid incidence in the cited papers corresponding to the numbers used in the recent study, and two of the four cited papers (2, 51) were of hospitalized patients from Wuhan in early 2020, so they are of limited usefulness in determining Long Covid prevalence.

In addition to the above incidence rates, the Northwestern study cited a GAO report that estimated “up to 23 million Americans suffered from long COVID.” The GAO report says: “Studies in the U.S. estimate that 10 to 30 percent of COVID-19 survivors develop long COVID. If so, 7.7 million to 23 million people in the U.S. may have developed long COVID as of February 2022.” However, there are no citations for the 10-30% number in that report.

The Northwestern study also cites a July 2022 letter from the American Academy of Neurology, which claims that Long Covid is “the third leading neurologic disorder in the US.” That claim was made based on an estimate that 30% of Covid survivors have Long Covid, and the letter has multiple references to both the GAO report and to the same Dr. Igor Koralnik who is a co-author of the Northwestern Long Covid study.

As you can see, many of these claims about Long Covid prevalence are of dubious accuracy and circular in nature. The widely-shared FOX32 Chicago article about the Northwestern study repeats the claim “Long Covid occurs in about a third of Covid survivors and is now the third leading neurologic disorder in the United States,” as if these were facts instead of a highly variable estimate, along with a hypothetical ranking based on that estimate.

Let’s Get Real

Recent serological data from the CDC estimates that 77.5% of Americans have had Covid. Long Covid estimates of ~30% would suggest that around 1 in every 4 Americans has either had or currently has Long Covid! Everyone should be able to acknowledge that’s simply not based in reality; however, these estimates continue to be repeated in academic papers and in the media. Some people seem to believe that a large portion of Americans are walking around suffering from post-Covid sequelae, often with debilitating neurological and pulmonary symptoms.

Most Long Covid studies are terribly flawed, and the reporting on them in the media and the discourse about them on social media makes the situation even worse. This study proves to be no exception. Unfortunately, these misleading studies serve to feed many people’s anxieties about Covid, worsening their mental and physical health. We need more responsible science and journalism on the topic of Long Covid, not more alarmism and hype.