11
$\begingroup$

After re-reading the definition of autocorrelation, is it fair to say:

  • All Time Series data have some autocorrelation? (either positive or negative autocorrelation, which are both bad)

There's probably some edge case that renders it false, but is it at least true 99% of the time?

"Each subsequent observation is created by taking the last observation and adding some random noise to it.

In a random walk time series, the data are positively autocorrelated: the data points near one another in time tend to be similar to one another (this can be observed quantitatively by correlating the series with a lagged copy of itself, and through tests such as the Durbin Watson)."

Source: https://engineering.atspotify.com/2023/09/how-to-accurately-test-significance-with-difference-in-difference-models/

$\endgroup$
7
  • 8
    $\begingroup$ In theory, you can think up a time series without autocorrelation, as Dave writes. In practice, everything is autocorrelated, as Ggjj11 writes. The question is how strong the autocorrelation is. Also, autocorrelation is absolutely not "bad", it can help us predict better, although much of autocorrelation is probably better captured as seasonality. $\endgroup$ Nov 25, 2023 at 6:58
  • 3
    $\begingroup$ If your data is tracking a learning process, for sure positive autocorrelation is good because it means that the learner doesn't forget everything already learned at every time point. "Good" and "bad" have a more relevant meaning in real life; I wouldn't attach them to the values that some parameters of formal models can take. $\endgroup$ Nov 25, 2023 at 11:53
  • $\begingroup$ Also consider this thread: stats.stackexchange.com/questions/427418/… $\endgroup$
    – Ggjj11
    Nov 25, 2023 at 12:56
  • $\begingroup$ If you dropped 'some' would that change the meaning? $\endgroup$ Nov 25, 2023 at 22:43
  • 1
    $\begingroup$ "autocorrelation is bad" ? uh... what? $\endgroup$
    – leonbloy
    Nov 26, 2023 at 18:23

5 Answers 5

10
$\begingroup$

The main reason for (auto)correlation is that the transition from one state to another one is not completely random, meaning that given a State Space of size $|S|$ the transition probability is non-uniform $p(X_{t+1}|X_t)\neq 1/|S|$. Whenever there is some dynamics favouring the transition from one state to another one, then there is (auto)correlation (neglecting exotic counter example processes, which may or may not exist). In physics (describing our world) there are a lot of equations describing system dynamics and therefore $p(X_{t+1}|X_t)\neq 1/|S|$ is true for a lot of phenomena.

If you insert this non-uniform transition probability in the definition of the autocorrelation you find $E[(X_t-E[X_t])(X_{t+1}-E[X_{t+1}])]\neq 0$.

PS: the example by Dave has a uniform transition probability and is a white noise process. :)

PPS: the more or less exotic counter examples (where $p(X_{t+1}|X_t)\neq 1/|S|$ and still $E[(X_t-E[X_t])(X_{t+1}-E[X_{t+1}])] =0$) from above can be constructed in the "usual way" with nonlinear effects, see e.g. last row in this figure https://commons.wikimedia.org/wiki/File:Correlation_examples2.svg

$\endgroup$
1
  • 1
    $\begingroup$ Isn’t my example a white noise process? $\endgroup$
    – Dave
    Nov 25, 2023 at 21:43
7
$\begingroup$

No, in the following sense. In finance, it is common to assume that autocorrelations are equal to zero for logarithmic returns on stocks. The assumption often turns out to be approximately true, in the sense that the measured autocorrelations are statistically indistinguishable from zero (that is, not statistically significant). They are not numerically equal to zero, though, as you never get a measured value of $0.000000$.

$\endgroup$
6
  • 1
    $\begingroup$ The estimated autocorrelation has associated confidence intervals which include the zero - expressed differently $\endgroup$
    – Ggjj11
    Nov 25, 2023 at 10:58
  • $\begingroup$ This answer seems not very accurate: "approximately true"? Numericaly equal? Most importantly, this is an experimental observation and all that the answer implies is that sometimes, an autocorrelation compatible with zero is measure. This can simply be because you do not have enough sensitivite to measure the deviation (as the comment elaborated), which doesn't mean that it is zero. $\endgroup$
    – Mayou36
    Nov 26, 2023 at 14:58
  • 1
    $\begingroup$ @Mayou36, I have provided a real-world example where the assumption of zero autocorrelation is widespread and could be justified. The four other answers contain only theoretical considerations or toy examples, so I though it would be nice to give a prominent real-world example. Regarding a proof that an empirically observed time series was generated by a process which has exactly zero autocorrelation, this is impossible. Thus, I think your criterion in the last sentence of your comment is unreasonably strict. $\endgroup$ Nov 26, 2023 at 15:30
  • $\begingroup$ The question is whether all timeseries have some autocorrelation. The answer says that, for some timeseries, no autocorrelation is measured, which does not say that there is none. The example is "nice" but if you compare it to the other answers, it is definitely incomplete (i.e. the practical example could be added where there might be no autocorrelation). $\endgroup$
    – Mayou36
    Nov 26, 2023 at 18:47
  • $\begingroup$ @Mayou36, hmm, but an example where there might be autocorrelation would not answer the question, so why include one? $\endgroup$ Nov 26, 2023 at 19:01
5
$\begingroup$

Is it fair to say: "All Time Series data have some autocorrelation”?

NO

Here is an algorithm to produce a time series that lacks autocorrelation.

  1. At $t=1$, draw a point from a distribution, such as x1 <- rnorm(1, 0, 1) in R or x1 = np.random.normal(0, 1, 1) in Python.

  2. Do it again at $t=2$.

  3. Do it again at $t=3$.

…and so on for the duration of the time series…

If you follow this algorithm, you wind up with independent draws from the distribution across the time periods, so zero autocorrelation. However, this is still a time series.

$\endgroup$
3
  • 1
    $\begingroup$ You have specified the marginal distributions, but we need the joint distribution to answer the question. There are algorithms (implemented as functions in R) that generate independent random variables. $\endgroup$ Nov 25, 2023 at 10:22
  • 7
    $\begingroup$ What you described can be shortened to just one word "white noise". And it could just fall into the "1% edge cases" as OP mentioned. $\endgroup$
    – Zhanxiong
    Nov 25, 2023 at 11:39
  • 1
    $\begingroup$ @RichardHardy I think the joint distribution actually is specified; I agree with the assessment in the answer that "you wind up with independent draws" (up to quibbles about deterministic RNGs being imperfect approximations of true randomness). To get dependence, you would need to inject some additional processing to the algorithm that isn't stated here -- resulting in a different algorithm. $\endgroup$ Nov 27, 2023 at 5:34
4
$\begingroup$

I don't think it makes sense to say that any formal statistical model is "true" in reality; models are always idealisations, and what happens in reality is something different. Even the most iconic examples for statistical modelling like binomial for coin tossing idealise away the underlying physics, for which far more complicated models exist.

Now if you compare a simpler with a more complex model such as a model with zero autocorrelation with a certain time series model that allows for autocorrelation, it makes sense to think that "zero autocorrelaton" is an idealisation, and reality would have to try extremely hard to achieve exactly zero "true" autocorrelation, so if someone said "in reality nothing really has zero autocorrelation", I'd agree with this (of course $10^{-5}$ and all kinds of extremely close to zero numbers are technically nonzero).

That said, (a) in certain situations a model with zero autocorrelation can be a very good model for what happens ("good" being somewhat relative to what the model is used for, but one thing is that the data may look like typical data generated from such a model), (b) I wouldn't take any formal model involving restrictive assumptions such as independent innovations with nonzero autocorrelation as "really true" either; being "true" is just not the job of models. And then it's hard to even talk of any true single value of the autocorrelation, although often of course data can clearly rule out a zero autocorrelation model and be compatible with a model with larger (or negative) autocorrelation.

If you are prepared to accept a model that implies nonzero autocorrelation for your data that has (as any model) some restrictive assumptions that are not exactly fulfilled in your situation, you can well also accept a model with zero autocorrelation in a situation in which there is nonzero autocorrelation (as implied by a more complex model) if this is sufficiently close to zero. (I'm not saying this should always be done; it depends on the purpose of the model.)

By the way note that "time series data" isn't very restrictive as a definition. Any data that is collected over time can be seen as "time series data" even though in many cases a priori we may not really be interested in how they develop over time. This includes things such as coin tossing, the background noise of any process, or observations taken on individual items (such as persons) treated as independent that in fact have been taken at a certain time point, which was recorded. So pretty much anything you'd think of as independent, therefore not autocorrelated (by which I obviously mean that such a model is sensible but not that it's really precisely true, see above), can also be interpreted as "time series data" in principle, even though this may not have been intended by the question. If you say something like "99% of real time series have autocorrelation", this would depend on what kind of "time series" actually qualify for being counted, which isn't so clear or easy to define.

$\endgroup$
3
$\begingroup$

No

... probably a selection bias?

I assume you make additional assumptions about what a time series is.

A timeseries is a dataset, where time is an additional variable.

You can draw $N$ random numbers that have no autocorrelation (loop, lottery, ...). Now you add a timestamp to it of when they were drawn. You've got a "timeseries" with no autocorrelation.

Good examples are repeated experiments that have the same initial state in a lab. Technically, you can put on a timestamp and call it a "timeseries", but it doesn't add anything.

Selection bias

As mentioned in other answers, it's all about the (auto)correlation of states. If you look at timeseries, they are very often correlated because you only take the time into account if there is a meaningful correlation to be expected. That's why "most" timeseries that are used as such have a correlation.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.