Intuition of Random Walk having a constant mean

Question

I am very new to time series analysis.

A random walk is defined as $Y_t=\phi Y_{t-1}+\varepsilon_t$, where $\phi=1$ and $\varepsilon_t$ is white noise. It is said that process is non-stationary for its variance is not constant. However, the mean is constant.

I am having a hard time understanding the "mean is constant" because when I plot a random walk process in R, I can see the variance is clearly changing, but to me the mean is also changing because there is a trend.　

What does it exactly mean when they say random walk has a constant mean?
There are mathematical function that proves it but I want to understand more intuitively.

Really appreciate your help.

set.seed(1)

TT <- 100
y <- ww <- rnorm(n = TT, mean = 0, sd = 1)
for (t in 2:TT) 
{
  y[t] <- y[t - 1] + ww[t]
}

Answer 1

There is a difference between unconditional mean and conditional mean, as there is between unconditional variance and conditional variance.

Mean

For a random walk $$ Y_t=Y_{t-1}+\varepsilon_t $$ with $\varepsilon_t\sim i.i.d(0,\sigma_\varepsilon^2)$, the condtional mean is $$ \mathbb{E}(Y_{t+h}|Y_{t})=Y_t $$ for $h>0$. This means that given the last observed value $Y_t$, the conditional mean of the process after $h$ periods, $\mathbb{E}(Y_{t+h}|Y_{t})$, is that value, regardless of how much time $h$ has passed. If time starts at $t=0$, then we have the mean conditional on the initial value being $\mathbb{E}(Y_{h}|Y_{0})$. From this we can see that the conditional mean varies with the conditioning information but not the time differential $h$.

Meanwhile, the unconditional mean at any fixed time point $h$ is zero: $$ \mathbb{E}(Y_{h})=\mathbb{E}(\sum_{i=0}^h\varepsilon_i)=\sum_{i=0}^h\mathbb{E}(\varepsilon_i)=\sum_{i=0}^h(0)=0. $$ Since it does not vary with $h$, we could say the mean of the process is zero.

Variance

The conditional variance is $$ \text{Var}(Y_{t+h}|Y_t)=h\sigma_\varepsilon^2. $$ For a fixed time differential $h$, the conditional variance is not increasing (the fluctuations are not getting wilder) over time, but conditional on some fixed time point the unconditional variance grows linearly with the time difference. Thus contrary to the conditional mean, the conditional variance does not vary with the conditioning information but does vary with (namely, grows linearly in) the time differential $h$.

Meanwhile, the unconditional variance at any fixed time point $h$ is the number of the time point $h$ times the variance of the increment term: $$ \text{Var}(Y_h)=\text{Var}(\sum_{i=0}^h\varepsilon_i)=\sum_{i=0}^h\text{Var}(\varepsilon_i)=\sum_{i=0}^h(\sigma_\varepsilon^2)=h \sigma_\varepsilon^2 $$ where the second equality uses the independence of the increments $\varepsilon_i$. Note that we can easily define the variance at a fixed time point but it is not as simple otherwise. Without being very rigorous, one could say the variance is undefined for an undefined time point. (This is in contrast to the mean.)

Answer 2

To see what is happening you need more than one realisation of the random walk, because the mean and variance are summaries of the distribution of the walk, not of any single realisation.

This code repeats your code to plot 20 random walks

set.seed(1)

ys<-replicate(20,{
TT <- 100
y <- ww <- rnorm(n = TT, mean = 0, sd = 1)
for (t in 2:TT) 
{
  y[t] <- y[t - 1] + ww[t]
}
y
})

matplot(1:100,ys,type="l",
  col=rep(c("black","grey"),c(1,19)), lwd=rep(c(2,1),c(1,19)),lty=1)

to give

Any single realisation of the random walk will randomly walk off up or down the graph. The entire cloud of possible random walks stays centered at zero and spreads out as time passes; some go up, some go down, some stay near the middle. The mean of the cloud stays at zero; the variance increases linearly with time.