Consider a regression model $Y_n=X_n\beta +\varepsilon$, where $X_n$ is a $n \times p_n$ matrix, and $\varepsilon=(\varepsilon_1,...,\varepsilon_n)'$ consists of independent and identically distributed variables with $E(\varepsilon_1)=0$ and $Var(\varepsilon_1)=\sigma^2$. Suppose that $\varepsilon_1 \sim N(0, \sigma^2)$ and suppose that $\hat \sigma ^2$ is the estimator of $\sigma^2$. Let

$$\hat \sigma ^2=\frac{\left\|Y_n-X_n\hat\beta\right\|^2}{n-p_n}.$$

Then, $\hat \sigma^2$ consistent for $\sigma^2$.

I can prove that $\hat \sigma^2$ is unbiased estimator for $\sigma^2$.

How can I continue to prove that $\hat \sigma^2$ consistent for $\sigma^2$?


Hint: \begin{align*} P(|\hat{\sigma}^2 - \sigma^2| > \epsilon) \le \frac{E[(\hat{\sigma}^2 - \sigma^2)^2]}{\epsilon^2} \end{align*} by Markov's/Chebyshev's inequality. Then take the limit.


Alternative hint:

\begin{eqnarray} \hat{\sigma}^{2}=\frac{Y_n'(I-P_{X_n})Y_n}{n-p_n}=\frac{\varepsilon'(I-P_{X_n})\varepsilon}{n-p_n}. \end{eqnarray} So, \begin{align} \hat{\sigma}^{2}=\frac{\frac{\varepsilon'\varepsilon}{n}-\frac{\varepsilon'X_n}{n}\left(\frac{X_n'X_n}{n}\right)^{-1}\frac{X_n'\varepsilon}{n}}{\frac{n-p_n}{n}}. \end{align}

This is a continuous function of three arguments \begin{eqnarray*} \frac{X_n'X_n}{n}&\to_p&?\\ \frac{\varepsilon'\varepsilon}{n}&=&\frac{1}{n}\sum_{t=1}^{n}\varepsilon_{t}^{2}\to_p?\\ \frac{X_n'\varepsilon}{n}&\to_p&?. \end{eqnarray*}

  • $\begingroup$ Thanks a lot. For this method, $X_i$ is random variable, but for my case, $X_i$ is fixed. My supervisor asked me not to use this method. He told me to prove unbiased estimator and then continue to prove the lim of variance. $\endgroup$ – Vivian Zhang Dec 1 '16 at 19:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.