Estimating the mean and variance of a stationary time series

Question

For a stationary time series can I use the sample mean as an estimate for the time series mean?

Also what is the recommended way to estimate the variance? The series is auto-correlated.

Answer 1

On the question about the mean - yes, I would have thought so. Stationarity means $E[Y_t]=c$ for all $t$ and the sample mean should estimate $c$.

On the variance (I hope this helps):

Say you have a stationary, zero mean AR(1) series:

$Y_t = \phi Y_{t-1} + e_t$

Where $\{e_t\}$ is white noise. Then if you take Var$(Y_t)$ you get:

Var$(Y_t) = \frac{\sigma^2}{1 - \phi^2} > \sigma^2 = $Var$(e_t)$

Say we have just observed $Y_{t}$ and are about to tick into time $t+1$, then we are in a conditional situation instead:

Var$(Y_{t+1}|Y_t)=E[(Y_{t+1}-E[Y_{t+1}|Y_t])^2|Y_t] $

$= E[(Y_{t+1} - \phi Y_t)^2 | Y_t]$

$= E[ e_{t+1}^2 | Y_t] = $Var$(e_t) = \sigma^2$

Thinking about it, it seems to me this is the right way to imagine the process, as it can only be constructed one step at a time since each step depends on previous values. Thinking about Var$(Y_{t+1})$ assumes we have all the instantiations of $\{Y_t\}$ at once, but they had to be laid down one-at-a-time, leading us to Var$(Y_{t+1}|Y_t)$.

Obviously, this is just for AR(1)...

Answer 2

If the time series is indeed stationary, you may take the sample variance. However, since your time series is auto-correlated, you may be more interested in the variance of the residuals.

Stationary means that the joint distribution of the time series is unaffected by time shifts. Thus, you can define a meaningful marginal distribution for a single point. However, since your time series is auto-correlated, the conditional distribution of a particular value given the preceding values may be quite different from that marginal distribution. The difference between the expected mean at time t, given the time series prior to t, and the actual value is called the innovation. Measuring the variance of the innovation will give you a better idea of how "noisy" the process is.

For instance, suppose your time series is given by $X_t = 0.999 X_{t-1} + \epsilon_t$, with $\epsilon_t \sim \mathcal{N}(0,1)$. The variance of $X$ would be $1000$, but, depending on what you're looking at, the variance of $\epsilon$: $1$, might be a better metric.