4
$\begingroup$

I am confused on why a simple trend process is not stationary. Consider the following process:

$Y_t = a + bt + \epsilon_t$

The variance is clearly constant. However, the mean $bt$ is dependent on $t$. When shifted in time, the mean only depends on the time interval and is independent of history. For example, $Y_{0,t}$ and $Y_{t,2t}$ would have the same mean and variance. So why is this process not stationary?

Secondly, if we now consider the following process:

$Y_t = a + \sqrt{t}\epsilon_t$

Assume $\epsilon_t$ is standard normal.

In this case, the mean is constant, however, the variance is dependent on $t$. However, in this case the variance is proportional to the time interval, which means $Y_{0,t}$ and $Y_{t,2t}$ would have the same mean and variance. So why is this process not stationary?

If you could explain it intuitively rather than definition/proof that would be helpful. My understanding of a stationary process is that the first two moments of the process (mean and variance) remain the same when shifted in time or space. I don't think I have the right understanding of stationary processes.

$\endgroup$
  • 1
    $\begingroup$ Should be moved to Cross Validated $\endgroup$ – SRKX Mar 29 '12 at 6:39
  • $\begingroup$ Why do your $Y_{0,t}$ and $Y_{t,2t}$ have two subscripts? $\endgroup$ – naught101 Mar 29 '12 at 8:43
  • $\begingroup$ 0,t indicates from 0 to t. t,2t indicates from t to 2t. The processes between these 2 intervals have same mean and variance, although dependent on t. $\endgroup$ – user862 Mar 29 '12 at 11:25
  • 3
    $\begingroup$ You seem to be confounding a process with its increments. The linear trend is not stationary but its first increment is. In your second example, the variance of the first increment is not proportional to the time interval: $\text{Var}[(a + \sqrt{t}\epsilon_t) - (a + \sqrt{s}\epsilon_s)]$ = $\text{Var}[\sqrt{t}\epsilon_t - \sqrt{s}\epsilon_s]$ = $t+s$ (assuming $t$ and $s$ nonnegative, of course). It depends on both the start and stop times. Obviously that's not stationary. $\endgroup$ – whuber Mar 29 '12 at 19:29
7
$\begingroup$

I think I nice way to get the intuition is to simulate 3 series for $t=0,...,500$ and plot them:

  1. Autoregressive Stationary Series: $A_{t}=0.05+0.95A_{t-1}+u_{t}$
  2. Random Walk with Drift: $R_{t}=0.05+1R_{t-1}+u_{t}$
  3. Explosive Series: $E_{t}=0.05+1.05E_{t-1}+u_{t}$

where $u_{t}$ is just some white noise, like iid $N(0,1)$.

Look at $A$ and $R$:

enter image description here

The theoretical mean of $A$ is $1$ (red horizontal line) and its standard deviation is $3.2$. The graph will deviate from that mean over time, but not too far. $R$ will look qualitatively similar to $A$ early on, but begins to drift apart in the middle, but converges towards the end. In theory, the unconditional mean and variance of $R$ do not exist, and you can see that in the graph.

Now plot all 3 series on a graph with the same scale.

enter image description here

Can you see how $E$ just makes the other two look like a straight line? The slope parameter in $E$ exceeds $1$ by 0.05, the same amount that it falls short for $A$, but what a difference it makes! Here, the average makes no sense at all.

The other point is that $A$ and $R$ look like the sorts of things we see every day, but we are bad at guessing which ones are stationary, especially with fewer data.

This is shamelessly plagiarized from Econometric Methods by Jack Johnston and John DiNardo, which is sadly out of print.

$\endgroup$
  • $\begingroup$ nice graphs. I understand first process is mean reverting while random walk and last process diverges. However, the mean reverting AR process and random walk may not converge as the random walk has a non deterministic path and might diverge away further with equal probability. $\endgroup$ – user862 Mar 29 '12 at 17:50
  • $\begingroup$ What I am unable to understand is the fact that "mean and variance remain the same when shifted in time" for stationary processes. In my case, the process has same mean and variance when shifted in time. So why is it not stationary? I think I am missing a simple point. If you look at my process with variance $t$, it is basically $t$ size increments of random walk $\endgroup$ – user862 Mar 29 '12 at 17:52
  • $\begingroup$ I am not sure that makes sense to me. It seems to me that at the very least, your first process does not have a fixed mean and your second does not have a fixed variance. That makes them non-stationary: the expectation grows/shrinks over time in a dependable way. You seem to be focusing on how the change in $Y$ from time $2$ to $3$ compares to the change from time $102$ to $103$, which is a different issue. $\endgroup$ – Dimitriy V. Masterov Mar 29 '12 at 22:18
  • 1
    $\begingroup$ @user862: Your statement is wrong the variance is not the same in the second example, when shifted in time. You are misunderstanding the phrase "shifted in time". In your second example: $Y_t = a + \sqr{t}\epsilon_t$, for the values near $t=1$, you variance is proportional to 1 ($a+\epsilon$), but near $t=5$, your variance is proportional to $\sqrt{5}$ ($a+\sqrt{5}\epsilon$). So the variance is the same with respect to the square root of time. But time is changing, so your variance is also changing. $\endgroup$ – naught101 Apr 15 '12 at 7:57
5
$\begingroup$

You answer your question yourself: Because stationarity implies both a constant variance, and a constant mean.

If either term is dependent on time, the process is not stationary. In your first example, the mean is dependent on time, and in the second, variance is.

$\endgroup$
  • $\begingroup$ yes that is definition. But what does it mean to be stationary and when does a process have constant probability distribution when shifted in time or space. $\endgroup$ – user862 Mar 29 '12 at 11:24
  • $\begingroup$ It means that at any point in time (or space) the variance is equal to the variance at every other point in space, and the same for the mean. The variance and the mean are two parameters that can define a probability density function, so saying that the PDF is constant is just the same as saying that the mean and variance are constant, and vice versa. The definition of stationarity is the meaning of stationarity. If that doesn't answer your question, you'll have to be clearer. $\endgroup$ – naught101 Mar 29 '12 at 12:17
  • 3
    $\begingroup$ @user862: I believe stationarity's intuitive meaning is that the data looks the same at any point in time: if someone hands you two graphs of the data at different points in time -- say with the x-axis blacked out -- you won't be able to tell which one came first. $\endgroup$ – Wayne Mar 29 '12 at 13:31
  • $\begingroup$ @Wayne thats the definition I understand. So in my case, although the mean and variance are dependent on $t$, they are same when shifted in time or space. $Y_{0,t}$, the process from $0$ to $t$ and $Y_{t,2t}$, the same process from $t$ to $2t$ have same mean and variance. So why is $Y_t$ not stationary? $\endgroup$ – user862 Mar 29 '12 at 17:39
  • $\begingroup$ @user862: How are the means the same when shifted in time? The mean for $Y_{t,2t}$ is either greater than ($b > 0$) or less than ($b < 0$) that of $Y_{0,t}$. $\endgroup$ – Wayne Mar 29 '12 at 20:36
3
$\begingroup$

A process $Y_t$ is stationary when for any vector of times $(t_1,...,t_n)$ and for every time interval $\tau$ the joint distribution of the vector $$ (Y_{t_1},...,Y_{t_n}) $$ coincides with the joint distribution of the vector $$ (Y_{t_1+\tau},...,Y_{t_n+\tau}) $$ In your example, the distribution of the "vector" $$ (Y_1) $$ is normal (I'm assuming that the shocks $\varepsilon_t$ are normal in your example) with mean $a+b$ and variance equal to the variance of $\varepsilon_t$. On the other hand the distribution of the "vector" $$ (Y_2)=(Y_{1+\tau}) $$ where $\tau=1$ is normal with mean $a+2b$ and same variance. Therefore the process cannot be stationary. In the same way you prove that the second example you show is stationary (variance grows). For what we have said above you should see that a stationary process always has constant mean and variance. I think you are confusing stationary processes, e.g. AR(1), with a process with stationary increments, i.e. random walks.

$\endgroup$
  • $\begingroup$ You define strict stationarity. There is also weak stationarity about which OP seems to talk about. $\endgroup$ – mpiktas Mar 29 '12 at 8:44
  • $\begingroup$ @mpiktas: The OP's description is of something much weaker than weak stationarity. Didn't we see almost an identical question a few months back? $\endgroup$ – cardinal Mar 29 '12 at 10:46
  • $\begingroup$ @cardinal, this type of question belongs to group of questions which are asked from time to time and will be asked almost indefinitely. And yes OP only talks about constant variance and mean, which is weaker than weak stationarity if you want to name it. $\endgroup$ – mpiktas Mar 29 '12 at 11:00
  • $\begingroup$ The definition of stationarity as per wikipedia states "a process is stationary if it has constant conditional probability distribution when shifted in time or space". Could you please expound on this and how this translates to constant mean and variance. This will help me understand this concept better. $\endgroup$ – user862 Mar 29 '12 at 11:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.