4
$\begingroup$

Consider a time series process with a well-defined, finite unconditional variance. Given a realization of the process (a time series) and a model for it, there are at least two ways of estimating the unconditional variance:

  1. Use the model to derive an analytical expression of the unconditional variance as a function of error variance and model parameters, and substitute sample estimates for the theoretical quantities.*
  2. Use sample variance as if the data were $i.i.d.$**

Question: When should 1. be preferred over 2. and vice versa?

A partly overlapping question is this one.


I simulated some AR(1) processes and discovered that 1. and 2. give very similar results. The kernel densities of the estimated variances over 1000 repetitions are nearly identical, and this holds across different sample sizes and different values of the autoregressive coefficient.

I then simulated some GARCH(1,1) processes and found that 2. is a much more robust alternative. While the median estimates due to 1. and 2. over 1000 repetitions were almost the same, 1. had many more extreme values that were apparently due to occasionally poor estimation of the GARCH model parameters.

So far it looks as if the simpler alternative 2. fares quite well compared with a more sophisticated alternative 1...


*Example: for an AR(1) process, the unconditional variance is $\frac{\sigma^2}{1-\varphi_1^2}$ where $\sigma^2$ is the error variance and $\varphi_1$ is the autoregressive coefficient. To make the estimator operational, $\sigma^2$ and $\varphi_1$ have to be substituted by their sample counterparts due to, say, full maximum likelihood estimator for the AR(1) model.

** That is, $\widehat{\text{Var}}(X)=\frac{1}{n-1}\sum_{t=1}^T (x_t-\bar{x})^2$.

$\endgroup$
11
  • $\begingroup$ I wonder if sample size would make a difference here. Have you tried it for varying sample sizes? On one hand a small sample size would make (1) more bias (particularly for a GARCH type model), but it will also increase the variance of (2) (It will actually increase the variance of both estimators, but (2) will become particularly inefficient in a small highly correlated sample). In practice I would probably prefer (2) since I never know for sure what the true DGP is. But I would be concerned about the inefficiency of (2) if my sample size was small and may opt for (1) in such cases. $\endgroup$ Feb 2, 2016 at 20:42
  • $\begingroup$ @ZacharyBlumenfeld, I used a sample size of 100 for one of the GARCH simulations, and that yielded highly variable results for alternative 1. but was quite fine for alternative 2. Meanwhile, I tried different sample sizes with AR(1) but the results were always pretty similar; I tried samples as small as 30 and as large as 1000. $\endgroup$ Feb 2, 2016 at 20:53
  • $\begingroup$ I am not really able to answer this question. I do feel like it depends a lot on the model. For example, an AR (1) coeficient of .95 vs .2 will result in different amounts of variance for the second estimator while perhaps not effecting the first estimator so much. In practice there is also the issue of mispecification, where I may try to estimate an AR (2) model as an AR (1), making alternative 1 bias. In addition, deriving The standard errors of alternative 2 would be hard in practice...you would need something on the order of a bootstrap $\endgroup$ Feb 2, 2016 at 21:12
  • $\begingroup$ @ZacharyBlumenfeld, exactly, I was also thinking about the point of standard errors where 2. is at a disatvantage. $\endgroup$ Feb 3, 2016 at 6:07
  • $\begingroup$ Why don't you use nonparametric variance estimation, along the lines of the Newey-West estimator? $\endgroup$
    – Jeremias K
    Feb 3, 2016 at 10:03

2 Answers 2

2
$\begingroup$

The more population parameters one has to estimate, the worse. Moreover, the model may be misspecified.

On the other hand, the sample variance does not depend on the assumption of the elements of the process being independent, for its good asymptotic properties. We assume that these elements are homoskedastic, and this makes the sample variance a consistent estimator, irrespective of whether the elements are autocorrelated.

More formally, assume $\{X\}$ is an ergodic process for its mean, so $E(X)$ is constant and finite, and the sample mean of a realization of $\{X\}$ is a consistent estimator for $E(X)$ (by Kinchine's Theorem). By the properties of ergodicitiy, $\{Z\}\equiv \{X^2\}$ is also an ergodic process for its mean, and $W\equiv Z+b$ is also an ergodic process for its mean, for $b$ a constant. Set $b\equiv -(E[X])^2$. Then

$$E(W) = E(Z) -(E[X])^2 = E(X^2) -(E[X])^2=\text{Var}(X)$$

So $E(W)$ is consistently estimated by its sample counterpart, which is the sample variance of $\{X\}$.

$\endgroup$
4
  • $\begingroup$ So which estimator should be used for the unconditional variance of an AR process? I seem to remember reading somewhere (can;' remember the source now) that the estimator with Bessel correction (the one with $n-1$ in the numerator) is actually biased for AR and even more biased than the "uncorrected" estimator. Would you be able to opine on that? Thank you. $\endgroup$
    – Confounded
    Dec 14, 2021 at 19:01
  • 1
    $\begingroup$ @Confounded For $y_t = bt_{t-1} + v_t$ with the usual assumptions, and a sample of $T$ observations, we have $$E\left (\sum_{t=1}^Ty_t^2\right) = \sigma^2_v\cdot \Big[\sum_{i=0}^{T-1}b^{2i}+\sum_{i=0}^{T-2}b^{2i}+...+\sum_{i=0}^{0}b^{2i}\Big]$$ I guess you can take it from here. $\endgroup$ Dec 14, 2021 at 22:03
  • $\begingroup$ Thank you for your reply, but I am actually not sure how to "take it from here". Your expression gives the expectation of the sum of squared observations in terms of the unknown parameter $b$ so it's not clear to me how this can be progressed. Should we be using an estimate for $b$ in this expression to arrive at an estimator? Thank you $\endgroup$
    – Confounded
    Dec 14, 2021 at 22:53
  • 1
    $\begingroup$ @Confounded My comment responded to your question about the bias of the sample variance estimator. In my formulation, assuming $y_1 = v_1$ the expected value of $y$ is zero so the sum of squared observations is the sample variance (when divided by sample size or corrected so). Its expected value will tell us about the finite sample bias that it has. And no you do not need an estimate for $b$ to compute the theoretical expression. $\endgroup$ Dec 15, 2021 at 18:06
1
$\begingroup$

One solution would be to use both estimators simultaneously in a GMM framework. This would have the benefit of minimizing your standard error beyond that of either estimator used alone. Furthermore, because the GMM model would be over-identified, you could test whether or not the two estimators are significantly different. If they are, it would suggest that you choose between the estimators* or that your model used in estimating (1) does not adequately fit the data.

At the very least, this will give you some insight into the stability of the estimators on a case by case basis.

*Your simulations would suggest that you choose estimator (2). This is ultimately a subjective decision, but given the guaranteed consistency of (2) and your simulations it would seem to be the reasonable choice.

GMM Estimator

Let $\theta_0$ be the true unconditional variance you wish to estimate. The two estimators you proposed in respective order are $\hat \theta_1$ and $\hat \theta_2$. You can obtain a third, more efficient, estimator $\theta^\dagger$ by minimizing the following quadratic: $$ \theta^\dagger=\min_{\theta}\bigg\{\begin{pmatrix} \hat \theta_1-\theta\\ \hat \theta_2-\theta \end{pmatrix}^T \hat W \begin{pmatrix} \hat \theta_1-\theta\\ \hat \theta_2-\theta \end{pmatrix}\bigg\} $$

Where $\hat W$ is the estimated inverse covariance matrix of $(\hat \theta_1,\hat \theta_2)$. Due to the non iid nature of the data, the variance of $\hat \theta_2$ (and the covariance) will have to be estimated with special care, whether that be through a block bootstrap technique, HAC standard errors, Newey-West, or whatever else you deem appropriate.

Over-Identification and the J-Test

Because the GMM estimate is over-identified you could perform a "J-test" with the following hypotheses: $$ H_0: \begin{pmatrix} \theta_1-\theta_0\\ \theta_2-\theta_0 \end{pmatrix} = \mathbf{0}\;\;\;\;\;\; H_1: \begin{pmatrix} \theta_1-\theta_0\\ \theta_2-\theta_0 \end{pmatrix} \neq \mathbf{0} $$ The test statistic would be $$J=\begin{pmatrix} \hat \theta_1-\theta^\dagger\\ \hat \theta_2-\theta^\dagger \end{pmatrix}^T \hat W \begin{pmatrix} \hat \theta_1-\theta^\dagger\\ \hat \theta_2-\theta^\dagger \end{pmatrix} \stackrel{d}{\rightarrow} \chi^2_1$$

which, as noted, has an asymptotic chi-squared distribution with 1 degree of freedom under the null.

Under the null hypothesis: $\theta^\dagger$ would be the most efficient estimator for the unconditional variance.

Under the alternative: there is a discrepancy between $\hat \theta_1$ and $\hat \theta_2$. Either one estimator is unstable or the model chosen to estimate $\hat \theta_1$ does not fit the data well. Given the findings of your simulations, this would suggest that you choose $\hat \theta_2$ to estimate the variance or choose a different model to obtain $\hat \theta_1$.

Related Tests

Another related test that you may consider using is the Hausman-Wu test. This test will only work if the variance of $\hat \theta_2$ is greater than that of $\hat \theta_1$. In this setting it can be used to test the consistency of $\hat \theta_1$. Rather than explain the test in full detail, I refer you to the Wikipedia page which explains the test statistic and asymptotic distribution.

I know this doesn't directly answer your question, but at least it gives you a way to check things and perhaps answer the question yourself on a case by case basis. In some cases, it may even help you form an improved, more efficient estimator.

$\endgroup$
0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.