I'm a little confused right now regarding the LLN "jump" from probability limits to expectations and variances/covariances:

Say we have a linear regression model of the form with $S$ observations:

$$ y = X\beta + \epsilon. $$


$$ \hat{\beta}_{OLS} = (X'X)^{-1}X'y$$


$$ \text{plim} \hat{\beta}_{OLS} = \beta + \text{plim} \left(\frac{X'X}{S}\right)^{-1}\frac{X'\epsilon}{S} = \beta + E(X'X)^{-1}E(X'\epsilon). \ \ \ \ \ \ \ (1)$$

Assuming $X \in F^{S\times1}$, i.e. a column vector, and the model was demeaned, the term simplifies to

$$ \hat{\beta}_{OLS} = \beta + \frac{Cov(x_s,\epsilon)}{Var(x_s)}. \ \ \ \ \ \ \ (2) $$

My question now is the following:

  • if I cannot demean my model (because I'm in a panel setting and have to show inconsistency of OLS first), I cannot write it in a form like (2) but have to stick to the expectation form (i.e. (1)), right?
  • $\begingroup$ What does LLN mean? and pllim? and demean?! What do you try to prove? What are the hypotheses? Is X a fixed design matrix or a random matrix? What do you know about $\epsilon$? $\endgroup$
    – Elvis
    May 20, 2016 at 9:42

1 Answer 1


You can. By using partitioned-regression results (F-W-L theorem), if the model includes a constant term,

$$y = \alpha + \mathbf x' \beta + u$$

then the "slope coefficients" are in practice calculated by OLS as

$$\hat{\beta}_{OLS} = (\hat {\tilde X}'\hat {\tilde X})^{-1}\hat {\tilde X}'\hat {\tilde y}= \beta + (\hat {\tilde X}'\hat {\tilde X})^{-1}\hat {\tilde X}'u$$

Where $\hat {\tilde X} = X - \bar X$ (sample mean) and for later use $ {\tilde X} = X-E(X)$. Again, here $X$ does not include a series of ones (but the model does). In other words, whether you demean a priori or not, OLS will demean the variables automatically to estimate the slope coefficients, if you also include a constant term in the model. If you multiply and divide the above by sample size, you get the multivariate analog of $(2)$ since the variables are now in sample mean-deviation form and so these are estimated variance and covariance matrices (and in $(2)$ you should use hats, by the way).


$$\text{plim}\big( \hat{\beta}_{OLS} -\beta\big) = \text{plim}\left (\frac 1n\hat {\tilde X}'\hat {\tilde X}\right)^{-1}\text{plim}\left(\frac 1n \hat {\tilde X}'u\right)$$

The Law of Large Numbers does not "jump from probability limits to expectations": it is the very essence of the Law that the probability limit is the expected value. Then (under the necessary conditions)

$$\text{plim}\big( \hat{\beta}_{OLS} -\beta\big) = E\left (\frac 1n\tilde X'\tilde X\right)^{-1}E\left(\frac 1n \tilde X'u\right)$$

Now the variables are in deviations from their true expected values, and these expected values are the true covariance matrices, and you can write for example,

$$\text{plim}\big( \hat{\beta}_{OLS} -\beta\big) = [ \text {Var}(\mathbf x)]^{-1}\cdot \text{Cov}(\mathbf x \cdot u)$$

which is the probability limit of the multivariate analogue of $(2)$.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.