Are COVID-19-Related Infections and Deaths Over- or Under-Reported?

Whether you believe that COVID-19-related estimates, specifically infections and deaths, are over- or under-reported depends almost entirely on where you get your news. This is an unfortunate result of these sources having diverging motives. Some aim to inform the public while others are advancing their own agendas. Discerning which is which can prove difficult, especially because many of the arguments made by nefarious individuals and organizations are based on select aspects of the truth.

These discrepancies between various sources of information arise because of two factors. In the first case, different sources interpret the same information in two different ways. In many cases, the articles that you read in the popular press are adapted from scientific journals or scientific sources by journalists. This process is beneficial in that it makes laborious and technical information easily intelligible to the average citizen. However, it puts the every-day consumer of information an additional degree of separation away from the original source. We’ve all played telephone at one point or another. The further you get away from the original source, the more distorted the message by the time it gets to you.

In the second, far more insidious case, data are selectively used or deliberately manipulated to suit a previously determined conclusion. This unfortunately occurs frequently in less trustworthy “news” sources. These sources of information will present an incomplete accounting of the data, using only those figures which suit their ends.

For purposes of planning at the federal, state, local, and individual level, it is absolutely essential that we have accurate accounting on COVID-19-related estimates of infections, hospitalizations, and deaths. Inaccurate assessments can induce states to reopen and can mislead individuals into relaxing precautions prematurely. Moreover, deliberate misrepresentation of the data can be used to deceive the public and build support for deleterious policies.

The public deserves to have an understanding of the scope of the COVID-19 epidemic in the US. That is why, in this article, we will epidemiologically analyze accountings of COVID-19-related infections and deaths. We will present the totality of information and provide an unbiased assessment of these estimates.

Testing

COVID-19 caught the United States wholly and devastatingly unprepared. Limited testing capabilities caused strict criteria for initial coronavirus testing. The unfortunate consequence was that the vast majority of COVID-19 cases went untested and uncounted [1]. In fact, by some estimates, the United States counted less than 7% of cases during March and April, the first crucial months of the pandemic in the United States [2]. By this measurement, the United States has experienced in excess of 33 million cases of COVID-19.

While testing capacity has improved, there are still limitations to wide-spread testing throughout the United States. In many states and communities, testing is still not available to those who need it [3]. When testing is available, there are barriers to receipt of a COVID-19 test, especially to those who are economically disadvantaged [3]. The combination of these factors has led to the total confirmed COVID-19 case count drastically underestimating the true COVID-19 case count. This is a generally accepted fact within public health circles, although you may not hear it being discussed in the popular press.

Arrival

COVID-19 was first identified in China on December 31st, 2019. The first case was confirmed in the United States on January 20th, 2020 [4]. These dates, however, only indicate when we were first able to identify the pathogen within our populations. They do not reflect the actual dates when the virus began spreading. There is now evidence to suggest that the spread of COVID-19 began much earlier in China and the United States [5-9].

Travel restrictions to and from China went into place in late January. However, travel restrictions to and from Europe went into place much later. This allowed the introduction of the SARS-CoV-2 virus into populations throughout the United States. There is evidence to suggest that COVID-19 was introduced to New York, the epicenter of the US epidemic, through European travel [10].

COVID-19 was present and spreading in the US before the necessary restrictions and precautions were put into place. Research from Northeastern University suggests that by March 1st there were already 28,000 infections circulating in five major cities: Boston, Seattle, Chicago, San Francisco, and New York. On March 1st, there were only 23 confirmed cases in those cities [7].

The novelty of the SARS-CoV-2 virus made it difficult to detect in the United States, leading to significant, unchecked circulation in the early months of the pandemic. We do not yet know the extent of the initial spread of the virus, but the effect is obvious—both cases and deaths were drastically undercounted.

Monitoring Fatalities

There is no centralized reporting center for COVID-19-related fatalities. As a result, we have to use death certificates in order to count COVID-19-related deaths. There is a wide degree of latitude in the way that cause of death is attributed to individuals, as will be described shortly. As a result, death certificates are notoriously inconsistent as the assignment of cause of death. This problem preceded the outbreak of COVID-19 in the US, but the novelty and pervasiveness of the disease has exacerbated it.

The process of counting coronavirus deaths with death certificates is described in great detail in a Scientific American article that you can read here [11]. I have summarized the process below.

In short, death certificates can be signed either by a physician, medical examiner, or a coroner. Physicians typically sign the death certificates of patients who die within hospitals, which make up the majority of COVID-19-related deaths. Medical examiners and coroners are independent officials who work for counties or cities. Death certificates are completed by a medical examiner or coroner if the death occurs outside of the hospital.

A standard death certificate lists an immediate cause of death, potential secondary causes of death or chain of events that led to death, and then contributing factors such as heart disease, high blood pressure, and others listed below. In many cases, indication of cause of death is based on the judgement of the medical professional. Autopsies can be more informative, but are conducted infrequently. Most importantly, this process is completely decentralized, with processes varying between states, counties, cities, and even from physician to physician. With no standardized reporting procedures, inconsistencies between death certificates are inevitable. The CDC does use advanced coding to improve interpretation of death certificates, but this process typically takes more than a year to complete [5].

There are additional complications associated with the novelty of the virus. While we can draw some comparisons with other coronaviruses and respiratory pathogens, we are still learning about the specific pathogenesis of COVID-19. For instance, we now believe that the SARS-CoV-2 virus causes blood clots. There have been younger individuals who have died inexplicably of strokes and heart attacks, tested positive for COVID-19, and had no history of respiratory symptoms. The evidence that COVID-19 causes blood clots suggests that COVID-19 was the cause of deaths in these cases [12]. Although these cases were tested for COVID-19 and likely documented as COVID-19-related deaths, there are likely many more that went untested and the role of COVID-19 will remain a mystery.

The CDC has released guidelines for how to attribute a death to COVID-19 [13]. These guidelines encourage using information from COVID-19 testing whenever possible. However, although testing capacity has improved over the course of the US epidemic, testing is still limited and prioritized for the living. To address this, the CDC does allow for deaths to be listed as possibly or presumably linked to COVID-19.

This method of classification has allowed states to parse out which deaths they report and which they do not. More than half of states currently only report confirmed COVID-19 deaths, which leads to under-reporting of deaths attributable to COVID-19 [14].

Excess Deaths

Given the potential biases introduced by the use of death certificates to estimate COVID-19-related fatalities, us epidemiologists actually recommend another metric to estimate the impact of COVID-19 on mortalities within the US. Using historical data, we can generally estimate the expected number of fatalities during a given time period. These are called expected deaths. Subtracting expected deaths from total deaths within a geographic region allows us to calculate excess deaths. These deaths are attributable to an outside influence, which, in this case, is COVID-19.

The CDC regularly produces data on excess deaths [15]. Unsurprisingly, there have been thousands of excess deaths in recent weeks. Often times, the number of excess deaths exceed the number of reported COVID-19 attributed deaths. This is due to several factors. Hospitals are expending a large amount of resources of COVID-19-related care, leaving fewer resources for other treatments and procedures. Individuals who otherwise may have been able to be saved died as a result. Additionally, Americans are afraid to go to hospitals due to the risk of contracting COVID-19. This could result in individuals not seeking care, resulting in unnecessary deaths [16]. Finally, individuals are dying from undocumented COVID-19-related complications. In many ways, this figure enables the most accurate assessment of the impact of COVID-19 on fatalities in the US. Although some of the scenarios above relate to deaths not caused directly by COVID-19, they would not have resulted had the pandemic not taken place.

From March 15th through May 16th, the US experienced a 23% increase over the number of expected deaths, which equates to 85,900 excess deaths. Reported COVID-19 deaths over the same time period were 67,299. The difference between these two figures is 18,601 deaths or 21.7% [17]. It is also important to note that these excess mortalities are artificially reduced by factors influenced by the pandemic. For instance, more people are quarantining within their homes, which leads to a reduction in deaths due to traffic accidents, injuries, and other factor associated with day-to-day life. In this case, those excess mortality figures may be even higher [5].

We cannot say whether the number of excess deaths is entirely attributable to COVID-19. However, we can say that the true number of COVID-19-related deaths is likely higher than the numbers that are being reported.

Rebekah Jones

Rebekah Jones is a former data scientist for the Florida Department of Health (FDH). Jones claims that she was fired because she refused to manipulate Florida COVID-19 data in order to ensure state adherence to reopening guidelines [18-20]. Her claims provide a unique insight into how states are representing and potentially politicizing data.

Since her dismissal from the FDH, Jones has designed and implemented her own dashboard to detail the COVID-19 outbreak in Florida. There are several key distinctions that are made between Jones’s dashboard and that of the FDH. Jones identifies 90,776 cases and 3,110 deaths compared to 80,676 cases and 3,018 deaths by the FDH [21, 22]. Jones employs different methodology for counting cases and deaths. For cases, Jones counts positive antibody tests, while the FDH does not. For deaths, the FDH does not count those who contracted COVID-19 in Florida and died in other states, while Jones does.

Jones further alleges that Florida’s COVID-19 dashboard is misleading the public with respect to the presentation of testing results. Florida’s state dashboard reports the total number of tests which come back positive or negative, while she reports the number of people who have tested positive or negative [21, 22]. Florida’s dashboard, as a result, makes it appear that a larger proportion of the population has received a COVID-19 test. The differences in these numbers are shocking. Regular testing (not including antibody testing) on Jones’s dashboard indicates that 908,676 Floridians have been tested for COVID-19, while Florida’s dashboard indicates that 1,485,759 tests have been administered, with 82,719 positive persons [21, 22]. By these metrics, the overall percent positive rate on Jones’s dashboard is 9.1% compared to 5.6% on the Florida dashboard.

These figures are substantially different. Both of these metrics have merit. The difference is that Jones clearly defines her methodology. You are able to clearly understand how she arrived at all of her measurements and you are able to validate them for yourself. The Florida dashboard on the other hand, presents no methodology. This leaves the public guessing as to what the FDH’s measurements represent.

I will not go as far as to say that the public is being deliberately mislead by the FDH. However, I will say that there is a shocking absence of clear methodology, definitions for key terms and assumptions, and adequate sourcing and availability of crude data. Without these tools, we are not able to evaluate the measurements that are being presented and the claims of those who are using those measurements. Without evaluation there can be no oversight. Without oversight, whoever can claim whatever they want without repercussions.

Jones’s dashboard can serve as a blueprint for the state of Florida and health departments around the country. She provides detailed methodology, data definitions, and extensive sourcing.

Conclusion

We began with a question: are COVID-19 infections and deaths over- or under-reported? A thorough analysis of available information indicates that estimates of both COVID-19-realted infections and deaths are, in fact, underestimated. Using a clear methodological approach, we were able to explain why the number of confirmed infections drastically underestimate the number of true infections in the US. Eventually, these suppositions can be confirmed with widespread antibody testing in representative populations. Only then will we know the true scope of the outbreak. The true scope of COVID-19-related fatalities, however, may never be known. There will simply be no way to retrospectively confirm the number of individuals who died from COVID-19 or COVID-19-related complications. We can only provide a best estimate, which may be in the form of excess mortality estimates.

In order to write this article, I pulled information from a number of different sources. When doing your own assessments of COVID-19-related news, or news in general, it is important that you do the same. Cross-check your information across multiple reliable sources. Ask yourself if your news source seems to be pushing an agenda. Above all, think critically! If all else fails, check-in with us here at Intelligent Speculation. We will be providing regular write-ups on the emerging COVID-19 pandemic and can assure you that they will be well sourced, accurate, and unbiased.