9
$\begingroup$

Intro / Background / Example

A recent article connecting pollen with covid-19 has gone viral this week.

Higher airborne pollen concentrations correlated with increased SARS-CoV-2 infection rates, as evidenced from 31 countries across the globe PNAS March 23, 2021 118 (12) e2019034118

The third figure in that article sketches a correlation, which is used in a remarkable way.

https://www.pnas.org/cms/10.1073/pnas.2019034118/asset/5987e6ff-5fb8-4763-8a51-3e10e4cefcf1/assets/images/large/pnas.2019034118fig03.jpg Fig. 3 Bag plot depicting the date of onset of SARS-CoV-2 exponential infection phase. Date of onset of the exponential infection phase (x axis) across all sites versus the average pollen concentration of the previous 4 d (y axis).

It shows a (weak) correlation between pollen and time. We see that later in the month March there have been more higher pollen concentrations than earlier in the month March.

The remarkable thing about this correlation is that the time points have been chosen by some measure for the onset date of the covid-19 epidemic in various places (which happened around 13 March for this sample).

Due to this, the authors argue that there is some relation between the onset date of the covid-19 epidemic and pollen concentrations (which is subtly different from a relation between time and pollen concentrations).

On a cross-sectional design for all 80 regions under study, it was found that the onset date of the exponential phase per region positively and significantly correlated with the cumulative amount of pollen up to 4 d before (P < 0.001, r = 0.25)

However, the onset date has nothing to do with the found correlation. We can see this when we plot all the time series entirely and with the points from the onset day in Fig. 3 overlayed.

overlay of Fig 3 with additional data

The onset dates have little to do with the pollen concentrations and any other random selection/filter of time points around 13 March would have likely made a positive correlation because there are more and higher pollen peaks later in March than at the beginning of March.

In fact, if pollen do have an effect, then a positive correlation with onset date should actually be argued to indicate that higher airborne pollen concentrations reduce the SARS-CoV-2 infection rates, as the later onset dates for regions with more pollen indicate that it took longer before the epidemic has grown to some level. The article completely reverses the potential meaning of the correlation in Figure 3.

Question

This link between the time points (the onset dates) and the pollen concentration is a non sequitur.

Is there for this particular fallacy, with the correlation of time points, a specific name? Or is there a text book reference that demonstrates this fallacy?

For instance, if I would like to shorten the above story/explanation and just say a single sentence like "In figure 3 they make the error/fallacy of .... " What name or textbook reference could we place on the points?

$\endgroup$
9
  • $\begingroup$ In addition, further on in the article, they are sort of over-fitting the data, but it is "significant" because of low computed p-values. But it's underestimated because the computation of the p-values is based on random Gaussian data. Yet, they do not have random Gaussian data and instead they relate time series for different places. There will be correlations between different nearby places and also neighbouring time points will be correlated. Is there a name for that type of underestimating p-values due to correlated, non-independent, data like in time series and multiple related units? $\endgroup$ Mar 12, 2021 at 16:11
  • $\begingroup$ Not a purely statistical type, but this sounds like the general fallacy of apophenia. From a technical standpoint, it sounds like several statistical fallacies are committed to compounding effect, among them en.m.wikipedia.org/wiki/Clustering_illusion, en.m.wikipedia.org/wiki/Data_dredging, en.m.wikipedia.org/wiki/Publication_bias#Definition $\endgroup$ Mar 16, 2021 at 21:03
  • $\begingroup$ citeseerx.ist.psu.edu/viewdoc/… just one question before I start researching, is this case not a type of number 32 like here in the document a flaw in 'analysis of evidence'? p. 128 ff., I have the feeling that this is a spurious regression and they made a false derivation because of a lurker variable or correlation $\endgroup$ Mar 16, 2021 at 21:13
  • $\begingroup$ I wanted to add non sense correlation statpics.blogspot.com/2014/05/nonsense-correlations.html $\endgroup$ Mar 16, 2021 at 21:19
  • 1
    $\begingroup$ Ok i did some research and will provide my results for you in the next minutes hope it helps in some way $\endgroup$ Mar 17, 2021 at 11:20

4 Answers 4

11
+150
$\begingroup$

Observation of "spurious correlation" for time-series over the same time period is something that has been recognised in the statistical community for over a century. Yule (1926) has observed that comparison of time-series vectors breaches the usual independent sampling assumptions in statistical problems, and that some simple deterministic series lead to correlation values with non-zero magnitute --- in some cases giving perfect positive or negative correlation. Wald argues that when time-series have systematic serial correlation (i.e., auto-correlation) then they will tend to be correlated with one another when taken over the same or similar time periods, even if there is no causal connection between the series.

Below I give some simple examples that illustrate the phenomenon of interst here. For an affine time-series with non-zero slope, any time vector is perfectly correlated with its corresponding time-series vector. For out-of-phase sinousoidal time-series, the time-series vectors are strongly negatively correlated, and can be perfectly negatively correlated for particular time vectors. Of particular interest here is the first case, which shows the statistical relationship between a time vector and its corresponding time-series vector under a simple trend. The case in your question is similar, insofar as it looks at the correlation between time values and pollen concentrations at those times. The low positive correlation simply means that there is a slight increasing trend in pollen concentration (relative to its variance) over the period in which the time values of interest occur. As you correctly point out, this does not really mean much --- just that pollen concentration was trending upward (very weakly) over a particular time period that coincided with the onset of Covid phases.

All of this really just reflects the fact that contemporaneous trends in time-series vectors lead to correlation between those vectors. If two time-series trend in the same direction over the same time period then they will tend to be positively correlated over that period. Likewise, if two time-series trend in opposite directions over the same time period then they will tend to be negatively correlated over that period. Several examples can be seen in the book Spurious Correlations, where contemporaneous temporal trends lead to high correlation.

The fallacy that encapsulates your concern here is cum hoc ergo propter hoc ("with this, therefore because of this"). Inferring a causal connection from the mere fact that two things have contemporaneous trends can lead to error, and usually we require more than this for a good causal inference. (And certainly we would at least want to know if the authors here were testing a pre-registered hypothesis, or just making a post hoc observation of correlation. It is almost certainly the latter.) The take-home here is that when you observe that two time-series are correlated (even highly correlated) that does not really mean much, especially as evidence for an underlying causal connection. As you observe in your question, the correlation observed in the paper occurs because there was increasing pollen count during March, and that conincided temporally with more frequent onset of Covid "phases". That is really not saying much, and if you just said that plainly then it would be an unremarkable statement that would not suggest any causal link between the two things.


Perfect positive correlation: As a simple illustration of high positive correlation, consider an affine time-series of the form:

$$X_t = \alpha + \beta t \quad \quad \quad \beta \neq 0.$$

Suppose we take some time vector $\mathbf{t} = (t_1,...,t_n)$ and form the corresponding vector $\mathbf{x} = (x_1,...,x_n)$ composed of values of the series at those time points. Since $x_i = \alpha + \beta t_i$ for all $i = 1,...,n$ it is easy to show that these vectors are perfectly correlated ---i.e., they have Pearson correlation equal to one.


Strong/perfect negative correlation: As a simple example of high negative correlation, consider the two time-series of the form:

$$X_t = \sin (2 \pi \beta t) \quad \quad \quad Y_t = \cos (2 \pi \beta t) \quad \quad \quad \beta \neq 0.$$

Suppose we take some time vector $\mathbf{t} = (t_1,...,t_n)$ and form the corresponding vectors $\mathbf{x} = (x_1,...,x_n)$ and $\mathbf{y} = (y_1,...,y_n)$ composed of values of the series at those time points. Through the use of discrete Fourier transformation, it is easy to show that these vectors will tend to have high negative correlation, and in some cases they can have perfect negative correlation.


$\endgroup$
3
  • $\begingroup$ Nice Ben, those references are very useful. It seems to relate a bit more to the rest of the article though. (But that's still useful I will like to read Yule). Do you also have a pointer for the specific practice in Figure 3? What I was particularly looking for was the case when you have a correlation between some variable and time (which may or may not be spurious), and the selection of time points is done in a way (e.g. the onset of covid-19) such as to imply that this selection is somehow meaningful (covid-19 and pollen are related). $\endgroup$ Mar 18, 2021 at 5:57
  • $\begingroup$ and that conincided temporaly with more frequent onset of Covid "phases". The frequency of onsets does not matter much. We could shuffle those dates and still get a correlation. The issue is that the correlation of pollen with onset date is an irrelevant argument. So the later onset dates occured with a higher pollen count. So what? What does that mean? Maybe high pollen delayed the onset? That could be an alternative interpretation. They do not talk with many words about Figure 3 and what it means but the implication is, I guess, that pollen concentration has a relationship with the onset. $\endgroup$ Mar 18, 2021 at 6:26
  • $\begingroup$ @SextusEmpiricus: I can't really take credit for the reference (hat tip to Patrick Bormann for a link that mentioned the paper). I have not seen a paper that specifically examines the correlation between a time vector and its time-series, but I suppose that this is a special case of cross-correlation between time-series where one series is the identity function. I would recommend you have a look at literature on cross-correlation. Also, I have expanded para 2 to elaborate more on the present case. $\endgroup$
    – Ben
    Mar 18, 2021 at 22:31
5
+300
$\begingroup$

in addition to my article resource of different fallacies in academic research: http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.579.5429&rep=rep1&type=pdf

I screened one/two articles and a blog and a few other things, this is the main summary: Most of the research I found always use the term spurious or non-sense, as i mentioned in my comments.

The first resource I found dealt exactly with the danger behind these in terms of time series: The conclusion to the dangers is, that researchers seem to "not prewhitening" or in other words flatten the noise in a time series, to be sure the remaining parts could offer a small glance of a real relationship:

On prewhitening:

http://hosting.astro.cornell.edu/~cordes/A6523/Prewhitening.pdf

The article that deals with the dangers:

https://link.springer.com/content/pdf/10.3758/s13428-015-0611-2.pdf

Some excerpt:

We have shown clearly that cross-correlations between pairs of time series, or even pairs of series derived as averages of sets of series, can be misleadding. The key means of avoiding such spurious cross correlations is to prewhiten the series being cross-correlated. But even then, some spurious correlations may remain, and the results need to be treated with caution. Not only is critical interpretation necessary, but also an awareness that certain kinds of time series may not be appropriate for the prewhitening approach—for example, when data are binomial or the series show only sparse change

The article also deals with different methods to engage the danger, perhaps this is useful for you.

In addition I looked into the terms of causality, as the researchers you mention clearly draw some causal objectives from their observations.

I found this blog which is ourstanding in the term that it highlights, i dunno over 100!? papers and sources on causality and time series granger causality and so on: https://towardsdatascience.com/inferring-causality-in-time-series-data-b8b75fe52c46#4da2. Although i not completely read through all of it, as you can imagine. Perhaps you found something enlightening if my previous research is not sufficient, so taht you got at least another hint.

To summarize my findings a sentence like they make the error/fallacy of not checking for spurious correlations, or the error of not prewhitening or something like that, could be feasible, as long as you have a source that highlights the danger of not checking the data in depth behind two time series. Because it is not significant because it is in a certain area. We have to look at the series at a whole. I do not believe if it correlates on one or two time points you can make the inductive check that the whole series is like that or there is a causality. That should be also summed up under spurious.

However my insight does not deal completely with the fact, that the researchers left out some information from the beginning at the end of march (the pollen density), I believe this is the error of purely fraud or dunno. But if you believe the researchers made mistake. I would tend to look into the material I provided. Hope it helps in some way.

https://en.wikipedia.org/wiki/Oil_drop_experiment#Fraud_allegations

$\endgroup$
3
  • $\begingroup$ That's a lot of resources, thanks a lot, I will dig through them. One quick comment: The issue with graph 3 is, I believe, not so much a spurious correlation (although I agree it is in the rest of the article). (Here I am using spurious correlation as in 'confusing correlation with causation') The special thing about graph 3 is that even if there would be a causal relationship then the correlation might still turn out negative. The problem with the figure is that the correlation is one between time and pollen and has no logical link with some causal effect relating to covid-19. $\endgroup$ Mar 17, 2021 at 12:02
  • $\begingroup$ Your link made led me to 'how to lie with statistics' from Darrell Huff, and it describes 'the semi-attached figure', which seems to be a good reference. In this case the detached nature is very well disguised because some attachment seems present as the figures on the x-axis are the onset-times of the epidemic exponential phase. $\endgroup$ Mar 17, 2021 at 12:05
  • $\begingroup$ I'm happy that I could help you in some way. I Know what you mean maybe I did no express myself properly, but for me the logical link itself was eiher artifical thus a lie /because in reality it is missing), or pure random so I judged it as a whole. and yeah this could be a nice reference A semi-attached figure occurs when proof is given for a claim, but when the reader looks at it closely, the proof and the claim are not related. It’s called “semi-attached” (last senetence is from a source) I'LL remember our discussion and topic! Thanks Sextus! $\endgroup$ Mar 17, 2021 at 12:27
2
$\begingroup$

There are different concepts and some of them overlap. Also I think the main once are already been mention : ). I think these are also interesting in terms of time series and analysis.

  1. 'spurious regression' high R2 values and high t-ratios yielding results with no economic meaning. This could happed ussually in a) just silly correlations as in https://www.tylervigen.com/spurious-correlations, or see https://en.wikipedia.org/wiki/Spurious_relationship, or b) very common in time series that are not stationary, see Unit Roots here: https://en.wikipedia.org/wiki/Unit_root
  2. Correlation does not imply causation – Refutation of a logical fallacy. See https://en.wikipedia.org/wiki/Correlation_does_not_imply_causation. In many cases we not to better understand the situation, some useful tools could be Grander Causality (see https://en.wikipedia.org/wiki/Granger_causality) or the creation of experimental set up https://en.wikipedia.org/wiki/Design_of_experiments.
  3. Cum hoc ergo propter hoc ("with this, therefore because of this"). See https://en.wikipedia.org/wiki/Post_hoc_ergo_propter_hoc.
  4. Lucas Critique: https://en.wikipedia.org/wiki/Lucas_critique It is naive to try to predict the effects of a change in economic policy entirely on the basis of relationships observed in historical data, especially highly aggregated historical data. Kind of saying that it is impossible to predict the future in human systems, for instance when including policies.
  5. A self-fulfilling prophecy is the sociopsychological phenomenon of someone "predicting" or expecting something, and this "prediction" or expectation coming true simply because the person believes it will and the person's resulting behaviors align to fulfill the belief. See https://en.wikipedia.org/wiki/Self-fulfilling_prophecy. I think this is curious with the toilet paper during the beginning of Covid 19. Similar pattern can be observe in the stock market during economics crisis and bubbles.
$\endgroup$
2
  • $\begingroup$ Thank you for your answer. I agree that more can be said about the article as a whole and your overlapping concepts relate to that. Would you also have something about the specific issue mentioned in the question?... $\endgroup$ Mar 21, 2021 at 0:46
  • $\begingroup$ ...This is not about the spurious regression, but about the connection of some time points that are based on some variable X (which could have just as well been randomly chosen), to a non-spurious correlation of time and another variable Y, and then suggesting that X and Y have some relationship (pollen correlates with time here; that is not spurious; my issue is with the labeling of specific time points according to some criteria and then calling/suggesting this as a correlation of pollen with those criteria). $\endgroup$ Mar 21, 2021 at 0:48
1
$\begingroup$

This is not an answer providing a canonical example. However, this answer provides another occurrence of the same type of fallacy mistake in a slightly different context.

  • This makes it interesting to be placed as an answer. (I do not want to add it to the question which would make it too much cluttered)
  • It also shows that this type of fallacy is more widespread then just the single occurrence in the article about the correlation between pollen and covid in PNAS. A specific term for this specific fallacy might be desirable.

The following argument occurs in

Walrand, S. Autumn COVID-19 surge dates in Europe correlated to latitudes, not to temperature-humidity, pointing to vitamin D as contributing factor. Sci Rep 11, 1981 (2021). https://doi.org/10.1038/s41598-021-81419-w

COVID-19 surge date as a function of country mean temperature (A) and humidity (B) during the 2 preceding weeks and as a function of country PWC latitude (C), pointing to vitamin D as one of the primary factors (flags link countries between graphs).

The argument is that temperature and humidity do not correlate with the date and therefore are excluded as cause for the surge in COVID-19 cases. With temperature and humidity excluded this points to vitamin D (sunlight) as remaining factor.

However when we take the data from table 2 and plot sunlight then we get just as well a lack of correlation. This is illustrated in the graph below. In the first row, we see three graphs from the article. In the second row we see a fourth graph that could have been plotted (and would have shown that UV is similar to temperature and humidity) but has not been added in the article.

where is the 4-th plot

The fallacy is that they wrongly assume that there should be a correlation between temperature, humidity and/or UV-light and the onset date of the surge, if these factors play a role. However, the opposite is true. If temperature, humidity and/or UV-light play a role then you will expect that these factors are more or less similar on/before the day of onset in different countries.


By stating that the argument is a fallacy, I do not want to say that all the conclusions are wrong. There might still be a relationship with vitamin D. However, what we can say is that these data are not conclusive about vitamin D. In addition the data are no support for an argument that temperature and humidity do not play a role.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.