Take the 2-minute tour ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

How can I show that a random walk ($y$ follows a random walk) is not covariance stationary? I tried to work on the formula below (with no results) Could you give me just a hint on how to proceed please?

$$Cov(y_{t+h},y_t)=E(y_{t+h}\times y_t)-E(y_t)E(y_{t+h})$$

Is this approach right?

Important: $\epsilon_t$, the shock, is an iid sequence with mean $0$ and variance $\sigma^2_\epsilon$.


If $y$ follows a RW we have $$y_t=y_{t-1}+\epsilon_t$$

then,

$$Var(y_t)=Var(y_{t-1}+\epsilon_t)=Var(y_{t-1})+\sigma_\epsilon+2Cov(\epsilon_t,y_{t-1})$$ Now I see that the variance of $y_t$ depends on the variance of $y_{t-1}$. This should suggest me that we lack covariance stationarity.

share|improve this question
    
For example I know that $E(y_t)=y_{t-1}$, and also I know that its variance increases with time. –  Charlie May 23 at 10:04
1  
No, you know an exact expression for $y_t$, not its expectation. It's written in your question. Replace $y_t$ with something else in the covariance, the same way you did when you worked out the variance. –  Glen_b May 23 at 10:10
    
Incidentally, as a routine bookwork-type question, you should add the self-study tag and read its tag wiki info. –  Glen_b May 23 at 10:13
    
Now I see that $\text{Cov}(y_t,y_{t-1})=\text{Var}(y_{t-1})$ using the exact expression for $y_t$. In addition, I can argue that $\text{Cov}(y_s,y_{s-1})\neq \text{Cov}(y_t,y_{t-1})$ if $s\neq t$ because the $\text{Var}(y_{t-1})$ varies with time. If $y_0$ is not random, $\text{Var}(y_t)=t \sigma^2_\epsilon$. Am I right? Thank you for your help. –  Charlie May 23 at 10:25
1  
The point was to get to something like ... since $\sigma^2_\epsilon (t-s) >0$, $\text{Cov}(y_t,y_{t-1})> \text{Cov}(y_s,y_{s-1})$, ... –  Glen_b May 24 at 13:20

3 Answers 3

up vote 6 down vote accepted

I think you're making life hard for yourself there. You just need to use a few elementary properties of variances and covariances.

Here's one approach:

  • start with the algebraic definition of your random walk process.

  • derive $\text{Var}(y_t)$ in terms of $\text{Var}(y_{t-1})$ and the variance of the error term

  • show that $\text{Cov}(y_t,y_{t-1}) = Var(y_{t-1})$

  • argue that $\text{Cov}(y_s,y_{s-1})\neq \text{Cov}(y_t,y_{t-1})$ if $s\neq t$.

... though, frankly, I think even just going to the second step (writing $\text{Var}(y_t)$ in terms of $\text{Var}(y_{t-1})$ and the variance of the error term) is sufficient to establish it's not covariance stationary.

share|improve this answer
    
Thank you. Since the $\epsilon$ is iid, its covariance with $y_{t-1}$ is zero right? How can I show that the $Cov(y_t,y_{t-1})=Var(y_{t-1})$? –  Charlie May 23 at 9:59

For each integer $t$, $Y_t = \sum_{i=1}^t X_i$ where the $X_i$ are iid random variables. From the independence of the $X_i$, it follows that $\operatorname{var}(Y_t) = \sum_{i=1}^t \operatorname{var}(X_i) = t\sigma^2$. For integer $h$, let $W = \sum_{i=t+1}^{t+h} X_i$ and note that $W$ and $Y_t$ are independent random variables because they are functions (sums) of disjoint collections of independent random variables. Then, $$\begin{align} \operatorname{cov}(Y_{t+h},Y_t) &= \operatorname{cov}(Y_t+W,Y_t)\\ &= \operatorname{cov}(Y_t,Y_t) + \operatorname{cov}(W,Y_t) &{\scriptstyle\text{this step follows because the covariance operator is bilinear;}}\\ &= \operatorname{var}(Y_t) + 0 &\scriptstyle{\text{0 because independent RVs}~W~\text{and}~Y_t~\text{have zero covariance;}}\\ &= \operatorname{var}(Y_t)\\ &= t\sigma^2 \end{align}$$ and thus the covariance increases as a function of $t$ but is not a function of $h$ at all as is needed for covariance stationarity. More generally, you can show that $\operatorname{cov}(Y_t, Y_s) = \operatorname{var}\left(Y_{\min\{t,s\}}\right)$.

share|improve this answer
    
Thank you Prof... Where can I find the proof or at least some explanation about the fact that $\operatorname{cov}(Y_t+W,Y_t)=\operatorname{cov}(Y_t,Y_t) + \operatorname{cov}(W,Y_t)$ if $W$ and $Y$ are independent? And why do you say that they are independent? –  Charlie May 23 at 15:03
1  
@Charlie $\operatorname{cov}(Y_t+W,Y_t)=\operatorname{cov}(Y_t,Y_t) + \operatorname{cov}(W,Y_t)$ does not require independence: it is a property of covariance that holds generally. $W$ and $Y_t$ are independent because they are sums of two mutually exclusive sets of independent random variables. Generally, if $X$ and $Y$ are independent, so are $g(X)$ and $h(Y)$ for any (measurable) functions $g(\cdot)$ and $h(\cdot)$. If you do not already know these concepts. you are way in over your head when you look at covariance stationarity and random walks. –  Dilip Sarwate May 23 at 17:26
    
This is clean and clear (+1). For the demonstration to be logically complete you need to show (or at least assert the obvious fact) that the variance of $Y_t$ actually does change with $t$. –  whuber May 24 at 2:16
    
@whuber Thanks for the upvote. It occurs to me that even the requirement for identical distribution of the $X_i$ is not needed to prove that the random walk is not covariance stationary. Independence and nonzero variance (or at least one $X_i$ having nonzero variance) might suffice. But at that point we are somewhat far from the usual meaning of random walk. –  Dilip Sarwate May 24 at 2:37

A usual way that we show this is by writing the random walk as

$$y_t = \sum_{i=1}^tu_t$$ and so

$$\operatorname{Var}(y_t) = \operatorname{Var}\left(\sum_{i=1}^tu_i\right) = t\sigma^2$$

and

$$\operatorname{Cov}(y_t, y_{t+k})= E\left(\sum_{i=1}^tu_i\right)\left(\sum_{i=1}^{t+k}u_i\right)-E\left(\sum_{i=1}^tu_i\right)E\left(\sum_{i=1}^{t+k}u_i\right)$$

$$=E\left(\sum_{i=1}^tu_i\right)\left(\sum_{i=1}^{t}u_i+ \sum_{i=t+1}^{t+k}u_i\right) -E\left(\sum_{i=1}^tu_i\right)E\left(\sum_{i=1}^{t}u_i+ \sum_{i=t+1}^{t+k}u_i\right)$$

$$=E\left(\sum_{i=1}^tu_i\right)^2 - \left[E\left(\sum_{i=1}^tu_i\right) \right]^2 +E\left(\sum_{i=1}^tu_i\right)\left(\sum_{i=t+1}^{t+k}u_i\right) -E\left(\sum_{i=1}^tu_i\right)E\left(\sum_{i=t+1}^{t+k}u_i\right)$$

$$=\operatorname{Var}\left(\sum_{i=1}^tu_i\right) + \operatorname{Cov}\left(\sum_{i=1}^tu_i, \sum_{i=t+1}^{t+k}u_i\right)$$

The two sums in the covariance term are independent since the white noises in the first do not appear in the second (different time-indices), so this covariance is zero, and we are left with

$$\operatorname{Cov}(y_t, y_{t+k}) = \operatorname{Var}\left(\sum_{i=1}^tu_i\right)=t\sigma^2$$

share|improve this answer
    
Are you assuming that $E[u_i] = 0$ and so the result is applicable only to random walks without "drift"? I believe the result is true even when $E[u_i]\neq 0$. –  Dilip Sarwate May 23 at 23:12
    
@DilipSarwate It is not I that I assume it - the assumption $E[u_i]=0$ is part of the question, so my answer was given with respect to that. Of course it holds for a non zero-mean, and in fact it will be exactly the same, since shifting the distribution does not affect variance/covariance. –  Alecos Papadopoulos May 23 at 23:29
    
Yes, I know that the assumption $E[u_i]=0$ is part of the question, and also that the result is true without the assumption of zero mean. But, your proof above does require the mean to be zero. Yes, your proof can be easily fixed to take into account nonzero means, and maybe you will re-write your proof when you have the time, but as it stands right now, it does need the mean to be $0$. –  Dilip Sarwate May 24 at 1:32
    
@DilipSarwate Well, I believe it is obvious, but let's trust your experience. I just generalized it. –  Alecos Papadopoulos May 24 at 2:07
    
@DilipSarwate Thanks, corrected. –  Alecos Papadopoulos May 24 at 2:24

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.