I was given the following problem involving OLS:

Suppose we have $(y_i,x_i,z_i)_{i=1}^n$ iid sequence, such that $x_i$ is a vector with K entries and $y_i$ and $z_i$ are scalars. Suppose $z_i$ is independent of $y_i$ and $x_i$, and that $E(y_i|x_i)=x_i'\beta$ for any $i$. Also, $E(z_i)=0$, $E(z_i^2)=2$. Define $w_i=z_ix_i$ and $q_i=z_iy_i$; $X=(x_1,...,x_n)'$, $Y=(y_1,...,y_n)'$, $W=(w_1,...,w_n)'$, $Q=(q_1,...q_n)'$.

What is the plim of $\gamma=(W'W)^{-1}W'Q$?

I guess it is probably $\beta$, but I could not find a way to rigorously show it. Any advice on how should I proceed? Many thanks!

  • $\begingroup$ The statement is unclear. If $Z$ is a true variable, then there is no need to introduce $\gamma$: the parameters are still $\beta_0$ and $\beta_1$. Also, it is suggestive to list all the observations. $\endgroup$ – Zhanxiong Mar 26 '16 at 16:12
  • $\begingroup$ Hey, I edited to show the original problem. Hope it's better! $\endgroup$ – LuizHS Mar 26 '16 at 17:42
  • $\begingroup$ I guess the definition of $q_i$ should be $z_iy_i$. Please double check it again. $\endgroup$ – Zhanxiong Mar 26 '16 at 17:50
  • $\begingroup$ Also, conventionally, the weak consistence needs some regularity condition such as $\frac{1}{n}X^TX \to Q$ for some positive definite matrix $Q$. $\endgroup$ – Zhanxiong Mar 26 '16 at 18:14
  • $\begingroup$ Thanks, error checked. Although the question did not impose regularity conditions, I assumed it so I could try some calculations. $\endgroup$ – LuizHS Mar 26 '16 at 19:12

You question can be rewritten as follows:

Given the general linear model: $$Y = X\beta + e \tag{1}$$ where $Y = (y_1, \ldots, y_n)^T, X = [x_1, \ldots, x_n]^T \in \mathbb{R}^{n \times K}, e = (e_1, \ldots, e_n)^T$. Suppose that $E[e \mid X] = 0$, $\text{Var}(e \mid X) = \sigma^2 I$, $x_1, \ldots, x_n$ are i.i.d. and $E[xx^T] = Q$ is positive-definite, prove that the LS estimator $\hat{\beta} = (X^TX)^{-1}X^TY$ converges weakly to $\beta$ as $n \to \infty$.

Notice that under the assumptions made for $(z_1, \ldots, z_n)$, multiplying both sides of $(1)$ by the diagonal matrix $Z = diag(z_1, \ldots, z_n)$ doesn't change the model specification of $(1)$. That is, $Y' = ZY, X' = ZX, e' = Ze$ again satisfy the dependence structure of $(1)$, so a proof to the statement in the box would be sufficient.

The result is classic, which can be verified by Chebyshev's inequality and weak law of large numbers, in detail, for any given $\varepsilon > 0$, by Chebyshev's inequality, it follows that \begin{align*} & P\left[\|\hat{\beta} - \beta\| \geq \varepsilon \right] \\ = & P\left[\|(X^TX)^{-1}X^Te\| \geq \varepsilon\right] \\ \leq & \frac{1}{\varepsilon^2}E\left[\|(X^TX)^{-1}X^Te\|^2\right] \\ = & \frac{1}{\varepsilon^2}E\left[\|(X^TX)^{-1}X^Te\|^2\right] \\ = & \frac{1}{\varepsilon^2}E\left[e^TX(X^TX)^{-2}X^Te\right] \\ = & \frac{1}{\varepsilon^2}E\left[\text{trace}(e^TX(X^TX)^{-2}X^Te)\right] \\ = & \frac{1}{\varepsilon^2} E\left[\text{trace}\left(ee^TX(X^TX)^{-2}X^T\right)\right] \\ = & \frac{1}{\varepsilon^2} \text{trace}\left(E\left[ee^TX(X^TX)^{-2}X^T\right]\right) \\ = & \frac{\sigma^2}{\varepsilon^2} \text{trace}\left(E\left[X(X^TX)^{-2}X^T\right]\right) \\ = & \frac{\sigma^2}{\varepsilon^2} E\left[\text{trace}\left(X(X^TX)^{-2}X^T\right)\right] \\ = & \frac{\sigma^2}{\varepsilon^2} E\left[\text{trace}((X^TX)^{-1})\right] \\ = & \frac{\sigma^2}{n\varepsilon^2} \text{trace}\left(E\left[\left(\frac{1}{n}\sum_{i = 1}^n x_ix_i^T\right)^{-1}\right]\right) \to 0 \end{align*} as $n \to \infty$. The last step is due to weak law of large numbers and may still need further technical conditions to ensure that the limit can be taken inside to the expectation. The proof can be a little less tedious if you would like to assume $X$ is nonrandom.

| cite | improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.