# Consistency of covariance matrix estimate in linear regression

Show that $\hat{\theta} =\frac{1}{N} \sum\limits_{i=1}^{N}\hat{u_i}^{2}x_i'x_i$ is a consistent estimator for $E(u^2x_i'x_i)$, by showing:

$$\frac{1}{N} \sum\limits_{i=1}^{N}\hat{u_i}^{2}x_i'x_i = \frac{1}{N} \sum\limits_{i=1}^{N}{u_i^{2}}x_i'x_i + o_p(1),$$

where $\hat{u}_i=y_i-x_i\hat\beta$ are the OLS residuals of the following regression model

$$y_i = x_i\beta + u_i,$$

where $x_i$ is a $1\times K$ vector, $\beta$ is a $K\times 1$ vector and $y_i$ with $u_i$ are scalars. We assume that $(y_i,x_i,u_i)$, $i=1,...,N$ forms an iid sample.

We should use the following hints:

1. $\hat{u}_i^2$ = $u_i^2 - 2u_ix_i(\hat{\beta}-\beta) + [x_i(\hat{\beta}-\beta)]^2$

2. $\hat{\beta} - \beta$ = $o_p(1)$

3. Sample averages are $O_p(1)$.

4. We assume all necessary expectations exist and are finite.

I'm getting stuck with this. I know I must be missing some simple sort of substitution or there is some gap in my knowledge or understanding preventing me from making the necessary manipulations.

I would love if someone could walk me through this and explain the intuition a bit here.

• Is it a homework problem? If so, tag it as [self-study]. – Tim Sep 8 '15 at 7:29
• There must be some sort of assumption in the background saying that $E(x'u)=0$, aka no correlation of error terms and regressors, aka "no misspecification". – Christoph Hanck Sep 8 '15 at 8:53
• I've added the missing information. This is an exercise from the Wooldridge book "Econometric analysis of Cross Section and Panel data". With the self-study tag this an appropriate question for this site, since it concerns proving asymptotic properties of certain linear regression statistics. – mpiktas Sep 16 '15 at 6:55
• I've made quite an extensive edits to the body and title, so that this question would be useful in future. – mpiktas Sep 16 '15 at 7:14
• I agree w/ @mpiktas: This question, as asked, meets our standards & should be considered on topic, IMO. – gung - Reinstate Monica Sep 18 '15 at 3:59

## 2 Answers

I am sorry I don't have any reputation to write it as a comment so I have to write it as an answer. First of all, you need to be careful with not forgetting to write $x_i'$ rather than $x_i$ in some instances, especially in 1.) I think by $\hat{\beta}$ Wooldridge means the OLS estimator of the parameter of interest in your regression equation, not what you denoted as $\hat{\beta}$ in the first line.

To solve this problem it is sufficient to assume several things: $$E[x_iu_i]=0$$ $$E[|x_{il}x_{im}|^2]<\infty \text{ for any } l,m$$ $$E[u_i^4]<\infty$$ under the condition that $\{(x_i,u_i)\}_{i=1}^n$ are i.i.d. across $i$.

Try using inequalities such as Cauchy-Schwarz or inequalities for matrix norms and then using laws of large numbers and Slutsky theorem.

• Nice answer. You shouldn't answer questions in the comments anyway! – shadowtalker Sep 10 '15 at 20:35
• The fourth moments are in fact non necessary. And the first condition is actually not necessary, since we have $\hat\beta=\beta+o_p(1)$. To prove this we need that condition naturally, but for this exercise it is not necessary. – mpiktas Sep 16 '15 at 7:10

Read the Wooldridge passage on proving that feasible GLS estimate is consistent. The proof relies on application on law of large numbers and Slutsky lema. It is actually not necessary to use the fact $Eu_ix_i=0$, as long as you have that $\hat\beta-\beta = o_p(1)$.

Using the hint number one insert the formula for $\hat{u_i}$ into left hand side of your main formula.

You will get three terms in a form $\frac{1}{N}\sum f(x_i,u_i,\hat\beta)$. Since $(x_i,u_i)$ form an iid sample, the law of large numbers gives you that $\frac{1}{N}\sum_{i=1}g(x_i,u_i) \to Eg(x_i,u_i)$.

Your main problem is that you need separate $\hat\beta - \beta$ from $x_i$ and $u_i$, only then you can use law of large numbers. For that use Slutsky lema. When you have separated $\hat\beta-\beta$, exploit the fact that it is $o_p(1)$, the fact that converging sequence (for which you have applied the law of large number) is $O_p(1)$ (the hint 3) and the fact that $o_p(1)O_p(1)=o_p(1)$.