# Pitfalls in time series analysis

I am just starting out self-learning in time series analysis. I've noticed that there are a number of potential pitfalls that aren't applicable to general statistics. So, building on What are common statistical sins?, I'd like to ask:

What are common pitfalls or statistical sins in time series analysis?

This is intended as a community wiki, one concept per answer, and please, no repetition of more general statistical pitfalls that are (or should be) listed at What are common statistical sins?

-

Extrapolating a linear regression on a time series, where time is one of the independent variables in the regression. A linear regression may approximate a time series on a short time scale, and may be useful in an analysis, but extrapolating a straight line is foolish. (Time is infinite and ever-increasing.)

EDIT: In response to naught101's question about "foolish", my answer may be wrong but it seems to me that most real-world phenomenon don't increase or decrease continuously forever. Most processes having limiting factors: people stop growing in height as they age, stocks don't always go up, populations cannot go negative, you can't fill your house with a billion puppies, etc. Time, unlike most independent variables that come to mind, has infinite support, so you really can imagine your linear model predicting Apple's stock price 10 years from now because 10 years from now will surely exist. (Whereas you wouldn't extrapolate a height-weight regression to predict the weight of 20-meter-tall adult males: they don't and won't exist.)

In addition, time series often have cyclical or pseudo-cyclical components, or random walk components. As IrishStat mentions in his answer, you need to consider seasonality (sometimes seasonalities at multiple time scales), level shifts (which will do strange things to linear regressions that don't account for them), etc. A linear regression that ignores cycles will fit over a short-term, but be highly misleading if you extrapolate it.

Of course, you can get into trouble whenever you extrapolate, time-series or not. But it seems to me that we too often see someone throw a time series (crimes, stock prices, etc) into Excel, drop a FORECAST or LINEST on it and predict the future via essentially a straight line, as if stock prices would rise continuously (or decline continuously, including going negative).

-
Can you expand on why it's foolish? –  naught101 Apr 26 '12 at 3:42
+1 for the awesome examples. I'm calculating exactly how many puppies I can fit in my house right now :D –  naught101 Apr 26 '12 at 6:25
This is a great illustration of your point: xkcd.com/605 –  Zach Apr 26 '12 at 13:45
@naught Mark Twain did a great job showing, in the plainest possible language, why "foolish" is appropriate to linear extrapolation of a time series. –  whuber Apr 26 '12 at 18:23
And this: stats.stackexchange.com/a/13904/9007 ... A similar point is extrapolation of a polynomial trend (especially of high degree), or any other model that doesn't have physical relevance. I wrote a blog post on that why this is a bad idea, when I was teaching myself octave. –  naught101 May 1 '12 at 0:28

Paying attention to correlation between two non-stationary time series. (It is not unexpected that they will have a high correlation coefficient: search on "non-sense correlation" and "cointegration".)

For example, on google correlate, dogs and ear piercings have a correlation coefficient of 0.84.

For an older analysis, see Yule's 1926 exploration of the problem

-
Of course not always. x<-seq(0,100,0.001); cor(sin(x)+rnorm(100001), cos(x)+rnorm(100001)) == 0.002554309 –  naught101 Apr 26 '12 at 6:22
@Wayne Thanks a ton for the Yule paper.I have been quoting this since 1970 and have never actually read it. It is well known in some, apparently small, circles. –  IrishStat Apr 28 '12 at 16:16

At the top level, Kolmogorov identified independence as a key assumption in statistics - without i.i.d assumption, many important results in statistics aren't true, whether applied to time series or more general analysis tasks.

Successive or nearby samples in most real-world discrete-time signals are not independent, so care must be taken to decompose a process into a deterministic model and a stochastic noise component. Even so, the independent increment assumption in classical stochastic calculus is problematic: recall the 1997 econ Nobel, and the 1998 implosion of LTCM which counted the laureates among its principals (though to be fair, the fund's manager Merrywhether likely is more to blame than quant methods).

-
"time series analysis" as a field of study. Basically I mean any thing that might trip up someone new to studying time series (of any type, and any type of analysis). I'm not looking for comprehensive answers. Check the question I referenced in my question to get a feel for what I'm trying to do here. –  naught101 Apr 26 '12 at 2:03
I meant what type of analysis –  alancalvitti Apr 26 '12 at 14:09
I know. I think you're missing the point of the question. Feel free to comment on common stumbling blocks in any type analysis, of any type of time series problem that you have experience with. Just keep it to problems that are specific to time series. –  naught101 Apr 27 '12 at 0:15
Hey @alancalvitti, that economics example sounds interesting. Do you know of a good description of it that we could link to from here? –  naught101 Apr 27 '12 at 14:04
I edited this answer to pare it back to the main point presented (to bring it back to one-point-per-answer style). That meant removing the stuff about spectral analysis. Perhaps something could said about that in a separate answer (although it didn't seem to be about pitfalls, specifically, there probably are spectral-analysis related pitfalls that we could note here). The above discussion now makes no sense, but you get that, I guess :/ –  naught101 Apr 27 '12 at 14:09

Being too certain of your model's results because you use a technique/model (such as OLS) that does not account for a time series' autocorrelation.

I don't have a nice graph, but the book "Introductory Time Series with R" (2009, Cowpertwait, et al) gives a reasonable intuitive explanation: If there is a positive autocorrelation, values above or below the mean will tend to persist and be clustered together in time. This leads to a less efficient estimate of the mean, which means that you need more data to estimate the mean to the same accuracy than if the there were zero autocorrelation. You effectively have less data than you think you do.

The OLS process (and therefore you) assume that there is no autocorrelation, so you are also assuming that the estimate of the mean is more accurate (for the amount of data you have) than it actually is. Thus, you end up being more confident of your results than you should be.

(This can work the other way for negative autocorrelation: your estimate of the mean is actually more efficient than it would be otherwise. I have nothing to prove this, but I'd suggest that positive correlation is more common in most real-world time series than negative correlation.)

-
An example here would be great, I don't fully understand the answer as it stands –  naught101 Apr 26 '12 at 23:36
Thanks for the edit @Wayne, but I kind of meant a real-world example, preferably with some visualisation. Obviously, others can add that too - it's a community wiki. –  naught101 Apr 27 '12 at 14:02
@naught101: Ah. Two of my three suggestions I've made here are based on what I've learned, but not necessarily well enough to make a good example. I'll try to look for one on the web. –  Wayne Apr 27 '12 at 15:00
It's only simulated data, but my answer to another question has some R code with a model fit with OLS and then more appropriately taking into account autocorrelation - with dramatically higher p-values. stats.stackexchange.com/questions/27254/… –  Peter Ellis Apr 28 '12 at 4:10

Defining Trend as a Linear growth over time .

Although some trends are somehow linear (see Apple stock price), and although time series chart looks like a line chart where you can find linear regression, most trends are not linear.

There are Step changes like changes when something happened in a specific point in time that changed the measure behavior ("The bridge collapsed and no cars is going over it since").

Another popular trend is "Buzz" - exponential growth and a similar sharp decline afterward ("Our marketing campaign was a huge success, but the effect faded after couple of weeks").

Knowing the right model (Logistic Regression, etc.) of the trend in the time series is crucial in the ability to detect it in the time series data.

-

The impact of level shifts , seasonal pulses and local time trends ... in addition to one-time pulses. Changes in parameters over time are important to investigate/model. Possible changes in variance of the errors over time have to be investigated. How to determine how Y is impacted by contemporaneous and lagged values of X . How to identify if future values of X can impact current values of Y. How to find out of particular days of the month have an impact. How to model mixed frequency problems where hourly data is impacted by daily values ?

naught asked me to provide more specific information/examples on level shifts and pulses. To that end I now include some more discussion. A series that exhibits an ACF suggesting non-stationarity is in effect delivering a "symptom". One suggested remedy is to "difference" the data. An overlooked remedy is to "de-mean" the data. If a series has a "major" level shift in the mean (i.e.intercept) the acf of this entire series can be easily misinterpreted to suggest differencing. I will show an example of a series that exhibits a level shift.If I had accentuated (enlarged) the difference between the two means the acf of the total series would suggest (incorrectly ! ) the need to difference. Untreated Pulses/Level Shifts/Seasonal Pulses/Local Time Trends inflate the variance of the errors obfuscating the importance of model structure and are the cause of flawed parameter estimates and poor forecasts. Now on to an example . This is a list of the 27 monthly values. This is the graph . There are four pulses and 1 level shift AND NO TREND ! and . The residuals from this model suggest a white noise process . Some (most !) commercial and even free forecasting packages deliver the following silliness as a result of assuming a trend model with additive seasonal factors. To conclude and to paraphrase Mark Twain. "There is nonsense and there is nonsense but the most non sensical nonsence of them all is statistical nonsense !" as compared to a more reasonable . Hope this helps !

-
Really? These are all pitfalls and sins? (Re-read the emphasized part of the question!) Perhaps you mean the opposite of what you wrote? –  whuber Apr 26 '12 at 18:26
The intent of my comments was to point out the pitfalls of not entertaining or considering some of these possible real-world structures. One needs to avoid assumptions that are not reasonably validated otherwise one might get some very questionable results. –  IrishStat Apr 26 '12 at 18:35
I gathered that was the intent, but in its current form your reply is easy to misunderstand. E.g., is it a sin to assess the "impact" of "one-time pulses" or is it a sin not to do that? This is sufficiently vague that a good case could be made for either interpretation! (Yes, it's a sin, because one-time pulses might just be outliers to which you don't want to grant undue influence and accounting for them all could overparameterize the model; no, they need to be included because their effects may linger for a long time and ignoring that could bias estimates of other parameters.) –  whuber Apr 26 '12 at 18:42
@whuber If the effects of a one-time pulse lingers this could be modeled as a sequence of one-time pulses at successive points. This is not as elegant as it might be but effective nonetheless. As you quite correctly stated you don't want to have errant values distorting parameter estimates of repetitive structure thus it is a "sin" to not to treat unspecified deterministic structure such as pulses,level shifts,seasonal pulses and/or local time trends. –  IrishStat Apr 26 '12 at 19:17
I think this'd be a really interesting answer if the first sentence (level shifts and pulses) was expanded on greatly (with some examples), and the rest was dropped. Heteroskedasticity would make for a good separate answer. –  naught101 Apr 27 '12 at 0:35

1. Failure to spot long cycles or seasonality - by examining only data over 'an insufficiently long' period of time
2. Failure to evaluate the forecasting error for past periods (backtesting)
3. Failure to detect and deal with regime changes

These problems are not related to the statistical methods involved but to the design of the study, i.e. which data to include and how to evaluate the results.

The tricky part with point 1. is making sure that we have observed a sufficient period of the data in order to make conclusions about the future. During my first lecture on time-series, the professor drew a long sinus curve on the board and pointed out that long cycles look like linear trends when observed over a short window (quite simple, but the lesson stuck with me).

Point 2. is especially relevant if the errors of your model have some practical implications. Among other fields, it is being widely used in Finance, but I would argue that evaluating the forecasting errors in past periods makes a lot of sense for all time-series models where the data allows it.

Point 3. touches again on the subject of which portion of past data is representative of the future. This is a complex topic with a large amount literature - I will name my personal favorite: Zucchini and MacDonald as an example.

-