I have time-series (daily) data and I want to understand the general trend.

My current approach is:

  1. Calculate the 7-day simple moving average.

  2. Add a line of best fit (linear least squares regression).

  3. Plot, then review metrics such as r, r^2, etc.

Question: is it good practice to draw a line-of-best fit on a moving average? I'm not very experienced but my understanding is MA and linear trend lines are both trend lines, so I'm not sure if it's OK to combine them in this way.

Raw data looks like this:

day             + count
2015-01-01 | 123
2015-01-02 | 290
2015-01-03 | 329
2015-01-04 | 276

Let me know if more detail would help- any direction on this is much appreciated.

  • 2
    $\begingroup$ It depends on how you compute step (2), "a line of best fit (linear least squares regression)." If you just drop the data into a least squares black box, most likely it operates under the assumption the errors are independent, whereas in a moving average the errors are strongly interdependent (e.g., neighboring 7-day averages have six days of data in common). You need to use a procedure that accounts for this. There are robust ways to explore trends, such as various nonparametric smoothers, so maybe it would be more fruitful to investigate them rather than fixing your current approach. $\endgroup$ – whuber Mar 16 '19 at 22:22

Of course you can do a fit on a moving average. That is your right. But the statistical diagnostics are not reliable anymore. The reason is that the IID property required in standard OLS are violated when you apply plain regression on a quantity that is highly autocorrelated.

Your $r^2$ will be artificially high and will insinuate a false sense of statistical significance. Think about this case, instead of a moving average do a linear fit on your original data in time first. And then do it a gain, you will get 100% $r^2$.

These models are not reliable ex-ante predictors and will have very low out-of-sample qualities.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.