For a stationary time series can I use the sample mean as an estimate for the time series mean?

Also what is the recommended way to estimate the variance? The series is auto-correlated.


On the question about the mean - yes, I would have thought so. Stationarity means $E[Y_t]=c$ for all $t$ and the sample mean should estimate $c$.

On the variance (I hope this helps):

Say you have a stationary, zero mean AR(1) series:

$Y_t = \phi Y_{t-1} + e_t$

Where $\{e_t\}$ is white noise. Then if you take Var$(Y_t)$ you get:

Var$(Y_t) = \frac{\sigma^2}{1 - \phi^2} > \sigma^2 = $Var$(e_t)$

Say we have just observed $Y_{t}$ and are about to tick into time $t+1$, then we are in a conditional situation instead:

Var$(Y_{t+1}|Y_t)=E[(Y_{t+1}-E[Y_{t+1}|Y_t])^2|Y_t] $

$= E[(Y_{t+1} - \phi Y_t)^2 | Y_t]$

$= E[ e_{t+1}^2 | Y_t] = $Var$(e_t) = \sigma^2$

Thinking about it, it seems to me this is the right way to imagine the process, as it can only be constructed one step at a time since each step depends on previous values. Thinking about Var$(Y_{t+1})$ assumes we have all the instantiations of $\{Y_t\}$ at once, but they had to be laid down one-at-a-time, leading us to Var$(Y_{t+1}|Y_t)$.

Obviously, this is just for AR(1)...

  • $\begingroup$ so if I had a series that appears stationary (say rejects ADF test of Ho of non-stationary) then how can I obtain just the residuals to estimate their variance? The word residuals also seems to have two meanings (1) The random variation (as opposed to deterministic trends) (2) the difference of an observed TS minus a fitted ARMA model - which should be uncorrelated white noise for a good fit? Thanks for the help, much appreciated! $\endgroup$
    – user23774
    Apr 1 '13 at 22:42
  • $\begingroup$ Hey - I'm just getting to grips with much of this stuff so not familiar with many specific tests. In practical terms if you build an ARIMA type model, say in R, the object will have the residuals in. In theoretical terms, the process variance Var($Y_t$) will depend on the particular model being used. Var($e_t$) is always going to be a generic $\sigma^2$. Personally I would associate 'residuals' with the empirical leftovers. $\endgroup$ Apr 2 '13 at 20:04

If the time series is indeed stationary, you may take the sample variance. However, since your time series is auto-correlated, you may be more interested in the variance of the residuals.

Stationary means that the joint distribution of the time series is unaffected by time shifts. Thus, you can define a meaningful marginal distribution for a single point. However, since your time series is auto-correlated, the conditional distribution of a particular value given the preceding values may be quite different from that marginal distribution. The difference between the expected mean at time t, given the time series prior to t, and the actual value is called the innovation. Measuring the variance of the innovation will give you a better idea of how "noisy" the process is.

For instance, suppose your time series is given by $X_t = 0.999 X_{t-1} + \epsilon_t$, with $\epsilon_t \sim \mathcal{N}(0,1)$. The variance of $X$ would be $1000$, but, depending on what you're looking at, the variance of $\epsilon$: $1$, might be a better metric.

  • $\begingroup$ Hi, thanks for answering. I'm new to time series so I'd appreciate it if you could expand on why I might be more interested in the variance of the residuals. I have started with (what appears to be a non-stationary series) and after differencing now appears stationary. So in this case are the residuals and why would the variance of those be of more use/interest than the stationary series as a whole? $\endgroup$
    – user23774
    Apr 1 '13 at 17:31
  • $\begingroup$ Ok think I'm starting to understand: its the innovations or 'shocks' at each time period that is causing the deviation from the expected conditional mean to the observed value? And the fact that we have auto-correlation tells us that the best estimate for the next time period is the expected mean conditional upon the previous observed value, rather than just the unconditional mean (say for completely uncorrelated series such as white noise?) $\endgroup$
    – user23774
    Apr 1 '13 at 23:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.