13
$\begingroup$

I am very new to time series analysis.

A random walk is defined as $Y_t=\phi Y_{t-1}+\varepsilon_t$, where $\phi=1$ and $\varepsilon_t$ is white noise. It is said that process is non-stationary for its variance is not constant. However, the mean is constant.

I am having a hard time understanding the "mean is constant" because when I plot a random walk process in R, I can see the variance is clearly changing, but to me the mean is also changing because there is a trend. 

What does it exactly mean when they say random walk has a constant mean?
There are mathematical function that proves it but I want to understand more intuitively.

Really appreciate your help.

set.seed(1)

TT <- 100
y <- ww <- rnorm(n = TT, mean = 0, sd = 1)
for (t in 2:TT) 
{
  y[t] <- y[t - 1] + ww[t]
}

enter image description here

$\endgroup$
13
$\begingroup$

There is a difference between unconditional mean and conditional mean, as there is between unconditional variance and conditional variance.

Mean

For a random walk $$ Y_t=Y_{t-1}+\varepsilon_t $$ with $\varepsilon_t\sim i.i.d(0,\sigma_\varepsilon^2)$, the condtional mean is $$ \mathbb{E}(Y_{t+h}|Y_{t})=Y_t $$ for $h>0$. This means that given the last observed value $Y_t$, the conditional mean of the process after $h$ periods, $\mathbb{E}(Y_{t+h}|Y_{t})$, is that value, regardless of how much time $h$ has passed. If time starts at $t=0$, then we have the mean conditional on the initial value being $\mathbb{E}(Y_{h}|Y_{0})$. From this we can see that the conditional mean varies with the conditioning information but not the time differential $h$.

Meanwhile, the unconditional mean at any fixed time point $h$ is zero: $$ \mathbb{E}(Y_{h})=\mathbb{E}(\sum_{i=0}^h\varepsilon_i)=\sum_{i=0}^h\mathbb{E}(\varepsilon_i)=\sum_{i=0}^h(0)=0. $$ Since it does not vary with $h$, we could say the mean of the process is zero.

Variance

The conditional variance is $$ \text{Var}(Y_{t+h}|Y_t)=h\sigma_\varepsilon^2. $$ For a fixed time differential $h$, the conditional variance is not increasing (the fluctuations are not getting wilder) over time, but conditional on some fixed time point the unconditional variance grows linearly with the time difference. Thus contrary to the conditional mean, the conditional variance does not vary with the conditioning information but does vary with (namely, grows linearly in) the time differential $h$.

Meanwhile, the unconditional variance at any fixed time point $h$ is the number of the time point $h$ times the variance of the increment term: $$ \text{Var}(Y_h)=\text{Var}(\sum_{i=0}^h\varepsilon_i)=\sum_{i=0}^h\text{Var}(\varepsilon_i)=\sum_{i=0}^h(\sigma_\varepsilon^2)=h \sigma_\varepsilon^2 $$ where the second equality uses the independence of the increments $\varepsilon_i$. Note that we can easily define the variance at a fixed time point but it is not as simple otherwise. Without being very rigorous, one could say the variance is undefined for an undefined time point. (This is in contrast to the mean.)

$\endgroup$
19
  • 1
    $\begingroup$ Indeed, the initial condition $Y_0 = 0$ nearly never makes sense in practice. So from a time-series point of view ($\neq$ maths) I would say that the unconditional expectation of a random walk does not exist and that the unconditional variance is infinite. As a general rule ARIMA models need "partially diffuse" initial conditions. $\endgroup$
    – Yves
    Apr 29 '21 at 8:15
  • $\begingroup$ @Yves, I share your sentiment. We can talk about the mean and variance at a fixed time point but not without specifying one. I will make that clearer in my answer. $\endgroup$ Apr 29 '21 at 8:17
  • $\begingroup$ @Yves, I have edited my answer. I think we disagree somewhat, and I welcome constructive criticism; I want to get a better grasp of this myself. Note that in my answer, I presume the random walk starts at time $t=0$. $\endgroup$ Apr 29 '21 at 8:26
  • $\begingroup$ As a general rule when using a random walk we require to have at least one observation, for an integrated random walk we similarly need two observations. For a general ARIMA model, we need a suitable number of obserations related to the differentiation order and a suitable observation design related to the differentiation operator (e.g. seasonal). Without these the expectation and variance do not make sense. Of course we can choose particular initial conditions but that is not what most time series software will do. $\endgroup$
    – Yves
    Apr 29 '21 at 8:31
  • 2
    $\begingroup$ @koyamashinji. the distribution of one coin toss is always the same. The distributions of a sum of multiple tosses where the numbers of tosses differ are different. The distribution of the sum approaches a normal one as the number of tosses approach infinity (hint: central limit theorem). A random walk is a sum of tosses, not a single toss. The variance grows linearly with the number of tosses. Also, we can meaningfully discuss variance of a fixed number of tosses but the discussion becomes problematic otherwise. $\endgroup$ Apr 29 '21 at 12:25
29
$\begingroup$

To see what is happening you need more than one realisation of the random walk, because the mean and variance are summaries of the distribution of the walk, not of any single realisation.

This code repeats your code to plot 20 random walks

set.seed(1)

ys<-replicate(20,{
TT <- 100
y <- ww <- rnorm(n = TT, mean = 0, sd = 1)
for (t in 2:TT) 
{
  y[t] <- y[t - 1] + ww[t]
}
y
})

matplot(1:100,ys,type="l",
  col=rep(c("black","grey"),c(1,19)), lwd=rep(c(2,1),c(1,19)),lty=1)

to give many random walks

Any single realisation of the random walk will randomly walk off up or down the graph. The entire cloud of possible random walks stays centered at zero and spreads out as time passes; some go up, some go down, some stay near the middle. The mean of the cloud stays at zero; the variance increases linearly with time.

$\endgroup$
1
  • $\begingroup$ From this we also have some evidence that the random walk is not ergodic. $\endgroup$ May 1 '21 at 23:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.