Everywhere in the news we hear about numbers of corona-virus infections, deaths and recoveries.

In lots of the publications the numbers (especially the infections) are compared with previous days. Mostly the presented numbers are 'growth' of new infections in per cent and the total number of infections so far. Then the numbers of different countries are shown.

I tried to get something out of these numbers (of infections) and realized that some important information is not told: the amount of tests done. And even further, who is tested? Which criteria? And this varies a lot between countries. Then there is the accuracy of tests.

The number and kind of tests done everyday change as well as more capacity is created or less capacity is available due to overloads or work restrictions.

I told myself the amount of deaths, as horrible as it sounds, is a good indicator of stabilization or regression. But no, people are not effected the same way regarding their previous medical condition. At some point in time, people at risk will "disappear" (again sorry for the word-choice).

I have the feeling that there so many variables and factors we are not able to gather regarding this pandemic (at least currently) that statistics are at their limit. Announced trends are most likely wrong.

How sure can we be about statistics in the current situation? Are trends viable? What can be compared what not? Is all current information foundationless?


Don't let the perfect be the enemy of the good

Many of the issues you have raised are perfectly reasonable concerns. Having said that, it is important to distinguish between cases where reported data is wrong, versus cases where reported data is correct, but is limited in its usage due to the omission of other relevant information. The latter case is really quite ubiquitous in statistical analysis, since it is very rare that we have all the data we would ideally like to have. In such circumstances, it is also important not to make the perfect the enemy of the good by presuming that the absence of a comprehensive dataset on every variable we would like to know about precludes any reasonable inferences being made from the data we have.

The main comprehensive public dataset we have available for the COVID-19 virus is the data held and updated by the John Hopkins Coronavirus Resource Centre at John Hopkins University. This is the repository of data that is being used for the vast majority of media reports and data visualisations on the spread of COVID-19. The database has data on confirmed infections, recoveries, and deaths, plus GIS data on where these occurred. The data is sourced from the WHO and various national health departments, and it is being updated frequently.

It is certainly true that there are a lot of other things we would ideally like to have data about, to assist in understanding the path and severity of the virus. As you have suggested, it would be wonderful if we could also obtaina comprehensive dataset on all the tests that have actually been conducted (including the negative results), and the characteristics of all the affected patients, including their age, sex, and health factors. At this early stage it is probably overly ambitious to expect all of this data to have been collected and collated, but hopefully the various health departments of the world will eventually be able to bring some of this data together. Health departments cannot legally disclose health information on particular patients in a way that would make them identifiable, so it is an extremely complex task to collate this type of data if the goal is to make it available for public analysis. No doubt there will be some efforts to obtain and collate more detailed data, but it will be a difficult task.

When interpreting limited data, it is desirable to describe that data in a way that makes it clear how it was collected, and this collection mechanism forms a caveat on analysis. Thus, we can refer to the reported numbers of infections, recoveries, deaths, etc., from each of the reporting countries, while noting that there may be disparities from these figures to the actual true numbers. Most health departments around the world are set up so that they can obtain these figures with reasonable accuracy, so that large diparities between reported figures and the true values are unlikely, except in the case of countries that make a deliberate effort to suppress this information. Notwithstanding the lack of the ideal data you would like, from the reported data there are certainly some things that we can reasonably infer. For most countries, the data is likely to be a reasonably representation of the number of infected, recovered, and dead patients out of those who have been properly tested. (Of course, I take some of the data, such as the recent numbers from China, with a large degree of scepticism.) It is also likely that there are other infected and recovered people who have not been tested, and who therefore do not form part of the reported statistics. Even so, this data gives us a pretty good look at growth rates for the infection in various cities and countries around the world, which allows us to see where the virus has progresssed most rapidly, and where it has been relatively contained. These numbers also allow some attempt at extrapolation, which allows health authorities to make predictions on the likely number of cases at future dates, and the health burden this will create in different places.

Your question appears to assert that most of these trends are likely to be "wrong" due to the absence of data on other factors such as total testing and patient health. In my view, that is not the correct way to look at it. Rather, the reported trends may be correct, in the limited descriptive sense, but there may still be underlying non-reported factors that would give reason to anticipate future changes in those trends. Personally, I am extremely impressed at how rapidly these various organisations have coordinated a large data collation and made public online platforms reporting on the infection and updating it frequently. I can log on every day and see updated graphical information showing the progression of the infection in virtually any city or country in the world! The data may not be perfect, but I would think it would bear enough of a resemblance to reality to be valuable information. I think the effort is quite amazing, and it far past what we could do even ten years ago.

  • $\begingroup$ "For most countries, the data is likely to be a reasonably representation of the number of infected, recovered, and dead patients." It is very likely that a large part of the infected people pass by unnoted. It requires a representative sample to assess a true prevalence of the virus/disease. The number of deaths and especially the number of recovered people is also dubious. When you take a ratio of the two and plot a time series then the numbers jump up and down and are all over the place. That is highly suscpicious (not expected) and should ring some alvrm bells. $\endgroup$ – Sextus Empiricus Mar 30 '20 at 21:54
  • $\begingroup$ What I meant was that this would be roughly true out of the pool of people who have been tested (i.e., when you limit interpretation to those that the data collection mechanism can capture). I have updated the answer to be more explicit about this. I agree that ratios against an unknown denominator are not reliable --- the growth rates I have in mind are those looking at growth in reported numbers of cases/deaths against known population figures. For example, I think you could reasonably say that growth of the infection has occured more rapidly in Spain/Italy than in Australia/NZ. $\endgroup$ – Ben Mar 30 '20 at 22:53
  • 1
    $\begingroup$ I agree, several indications can be made based on the numbers. But, I fear, numbers are being overinterpreted and placed out of context. I doubt the drivers for decisions/policies that are currently being made by governments. I wonder whether they are being driven by expert advice, or by public opinion. The latter, the public opinion (social media and Madonna), is in its turn mostly being governed by uncertainty and panic, but not by data-driven knowledge. Many governments have surrendered to public opinion. It seems like only Sweden, the country of Nobel, is listening to the specialists. $\endgroup$ – Sextus Empiricus Mar 31 '20 at 0:21
  • 1
    $\begingroup$ Sure, we should not make perfect become the enemy of good... but what is good about the numerous dashboards like the ones at the John Hopkins University website? Isn't this becoming more a hype rather than data science? $\endgroup$ – Sextus Empiricus Mar 31 '20 at 0:30
  • 1
    $\begingroup$ For Germany, wjst.de/corona has maps and plots. Its main point: "Central epidemiological data are also missing two weeks after the outbreak of the pandemic in Germany. This means that only a representation of the number of cases but NO FORECAST IS POSSIBLE." $\endgroup$ – denis Apr 3 '20 at 16:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.