Take the 2-minute tour ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

I understand that a stationary time series is one whose mean and variance is constant over time. Can someone please explain why we have to make sure our data set is stationary before we can run different ARIMA or ARM models on it? Does this also apply to normal regression models where autocorrelation and/or time is not a factor?

share|improve this question
1  
What are ARM models? Did you mean ARMA? –  mpiktas Dec 13 '11 at 10:00
4  
Stationarity requires more than constant mean and variance. Weak stationarity requires that covariance function $cov(X_t,X_{t+h})$ doe not depend on $t$. –  mpiktas Dec 13 '11 at 10:02
1  
You don't require stationarity to run an AR$I$MA model, since if the $I()$ order is $>0$, it's explicitly nonstationary. Stationarity is an assumption of ARMA, however. –  Glen_b Oct 20 '13 at 21:45
add comment

4 Answers

Stationarity is a one type of dependence structure.

Suppose we have a data $X_1,...,X_n$. The most basic assumption is that $X_i$ are independent, i.e. we have a sample. The independence is a nice property, since using it we can derive a lot of useful results. The problem is that sometimes (or frequently, depending on the view) this property does not hold.

Now independence is a unique property, two random variables can be independent only in one way, but they can be dependent in various ways. So stationarity is one way of modeling the dependence structure. It turns out that a lot of nice results which holds for independent random variables (law of large numbers, central limit theorem to name a few) hold for stationary random variables (we should strictly say sequences). And of course it turns out that a lot of data can be considered stationary, so the concept of stationarity is very important in modeling non-independent data.

When we have determined that we have stationarity, naturally we want to model it. This is where ARMA models come in. It turns out that any stationary data can be approximated with stationary ARMA model, thanks to Wold decomposition theorem. So that is why ARMA models are very popular and that is why we need to make sure that the series is stationary to use these models.

Now again the same story holds as with independence and dependence. Stationarity is defined uniquely, i.e. data is either stationary or not, so there is only way for data to be stationary, but lots of ways for it to be non-stationary. Again it turns out that a lot of data becomes stationary after certain transformation. ARIMA model is one model for non-stationarity. It assumes that the data becomes stationary after differencing.

In the regression context the stationarity is important since the same results which apply for independent data holds if the data is stationary.

share|improve this answer
    
+1 Intelligible reference of important basics –  ttnphns Dec 13 '11 at 12:27
add comment

What quantities are we typically interested in when we perform statistical analysis on a time series? We want to know

  • Its expected value,
  • Its variance, and
  • The correlation between values $s$ periods apart for a set of $s$ values.

How do we calculate these things? Using a mean across many time periods.

The mean across many time periods is only informative if the expected value is the same across those time periods. If these population parameters can vary, what are we really estimating by taking an average across time?

(Weak) stationarity requires that these population quantities must be the same across time, making the sample average a reasonable way to estimate them.

In addition to this, stationary processes avoid the problem of spurious regression.

share|improve this answer
add comment

Since ARIMA is regressing on itself for the most part, it uses a type of self-induced multiple regression that would be unnecessarily influenced by either a strong trend or seasonality. This multiple regression technique is based on previous time series values, especially those within the latest periods, and allows us to extract a very interesting "inter-relationship" between multiple past values that work to explain a future value.

share|improve this answer
add comment

Time Series is about analysing the way values of a series are dependent on previous values. As SRKX suggested one can difference or de-trend or de-mean a non-stationary series but not unnecessarily!) to create a stationary series. ARMA analysis requires stationarity. $X$ is strictly stationary if the distribution of $(X_{t+1},\ldots,X_{t+k})$ is identical to that of $(X_1,\ldots,X_k)$ for each $t$ and $k$. From Wiki: a stationary process (or strict(ly) stationary process or strong(ly) stationary process) is a stochastic process whose joint probability distribution does not change when shifted in time or space. Consequently, parameters such as the mean and variance, if they exist, also do not change over time or position. In addition as Cardinal has correctly pointed out below the autocorrelation function must be invariant over time (which means that the covariance function is constant over time) converts to parameters of the ARMA model being invariant/constant for all time intervals.

The idea of stationarity of the ARMA model is closely tied into the idea of invertibility.

Consider a model of the form $y(t)=1.1 \,y(t-1)$. This model is explosive as the polynomial $(1-1.1 B)$ has roots inside the unit circle and thus violates a requirement. A model that has roots inside the unit circle means that "older data" is more important than "newer data" which of course doesn't make sense.

share|improve this answer
    
A bit of a quibble: It is not quite clear what you mean when you say that "$X$ is second-order stationary if the first two moments are invariant over time." Normally, when I think of second-order stationarity, I think of the autocorrelation function being invariant over time in addition to the invariance of the mean. This is, of course, a (much) stronger condition than the (naive?) interpretation of the one you state. –  cardinal Dec 13 '11 at 1:23
    
The mention of second-order stationary seems to have been lost in your most recent edit. Was that intentional? (My original comment was more directed toward second-order stationarity than strict stationarity.) –  cardinal Dec 13 '11 at 2:17
    
:cardinal I guess I felt that your comment was important and made it clearer as to what was being assumed. If you think the idea of "second order stationary" adds clarity please help me add it to my answer in a way that sheds light in simple straightforward English. –  IrishStat Dec 13 '11 at 10:06
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.