Understanding and interpreting consistency of OLS

Question

Many econometrics textbooks (e.g. Wooldridge, "Econometric analysis...") simply write something similar to: "If the population model is $y = xB + u$ and (1) $\text{Cov}(X,U) = 0$; (2) $X'X$ is full rank, then OLS consistently estimate parameters $B$". I've worked out the math behind consistency but I'm a bit lost in interpreting the meaning.

We're talking about consistent estimation, but estimation of what? Please tell me if the following points are correct:

1. If we have random sample $X,Y$ and $X'X$ is invertible, then we can always define Best Linear Predictor of $y$ given $x$. And then OLS always consistently estimates coefficients of Best Linear Predictor (because in BLP we have $\text{Cov}(u,x)=0$ from the definition). Bottom line: we can always interpret OLS estimates as coefficients of BLP

2. The only question is whether BLP corresponds to conditional expectation $\text{E}(y|x)$. If it does (for which we need $\text{E}(u|x) = 0$), then we can interpret OLS estimates as partial effects.

What is wrong with this or what am I missing? I don't get the point of stating the assumption of $\text{Cov}(u,x) = 0$, if this assumption is by definition fulfilled in case of Best Linear Predictor. And if we want to give structural interpretation to BLP coefficients (partial effects) we need stronger assumption of $\text{E}(u|x) = 0$ anyway. Is there any other interpretation we may have with zero covariance?

Answer 1

An estimator is consistent if $\hat{\beta} \rightarrow_{p} \beta$

Or $\lim_{n \rightarrow \infty} \mbox{Pr}(|\hat{\beta} - \beta| < \epsilon) = 1 $ for all positive real $\epsilon$.

Consistency in the literal sense means that sampling the world will get us what we want. There are inconsistent minimum variance estimators (failing to find the famous example by Google at this point).

Unbiased minimum variance is a good starting place for thinking about estimators. Sometimes, it's easier to understand that we may have other criteria for "best" estimators. There is the general class of minimax estimators, and there are estimators that minimize MSE instead of variance (a little bit of bias in exchange for a whole lot less variance can be good). These estimators can be consistent because they asymptotically converge to the population estimates.

The interpretation of the slope parameter comes from the context of the data you've collected. For instance, if $Y$ is fasting blood gluclose and $X$ is the previous week's caloric intake, then the interpretation of $\beta$ in the linear model $E[Y|X] = \alpha + \beta X$ is an associated difference in fasting blood glucose comparing individuals differing by 1 kCal in weekly diet (it may make sense to standardize $X$ by a denominator of $2,000$.

That is what you consistently estimate with OLS, the more that $n$ increases.

WRT #2 Linear regression is a projection. The predictors we obtain from projecting the observed responses into the fitted space necessarily generates it's additive orthogonal error component. These errors are always 0 mean and independent of the fitted values in the sample data (their dot product sums to zero always). This holds regardless of homoscedasticity, normality, linearity, or any of the classical assumptions of regression models.

Answer 2

When we talk about consistent estimation, we mean consistency of estimating the parameters $\beta$ from a regression like $$y = \alpha + \beta x + u$$ $\newcommand{\plim}{{\rm plim}}\newcommand{\Cov}{{\rm Cov}}\newcommand{\Var}{{\rm Var}}$ We don't know the true value of the slope of $x$ in this linear model, i.e. we don't know the true value of $\beta$. This is why we estimate it in the first place. When you estimate this model and if OLS is BLUE, then $$\plim\: \widehat{\beta}_{OLS} = \beta $$ This means, as your sample size becomes larger and larger, your estimate $\widehat{\beta}$ converges to the true value $\beta$. In this case you are "consistently" estimating this parameter. If you had the entire population as a sample, you would get $\widehat{\beta} = \beta$

As concerns your (1) and (2), $\Cov(X,u) = 0$ is one of the requirements for an estimator to be best, linear and unbiased (BLU). This doesn't mean that every estimator fulfills these requirements. If $\Cov(X,u) \neq 0$, OLS is biased (but it may still be "best", i.e. it has the smallest variance, and it will be "linear"). Consider again a linear regression model: $$y = \alpha + \beta x + \gamma d + u$$ Suppose that $\gamma \neq 0$, $\Cov(x,d) \neq 0$, and that $d$ is missing from the regression, so we only regress: $$y = \alpha + \beta x + e$$ This means that the $d$ is included in the error: $e = u + \gamma d$, and because $x$ is correlated with $d$, our OLS estimator is not BLUE anymore because $\Cov(x,e) \neq 0$ (since $d$ is inside $e$). Therefore, our estimate $\widehat{\beta}$ will be biased and inconsistent with $$\plim\: \widehat{\beta} = \beta + \gamma \frac{\Cov(x,d)}{\Var(x)}$$

As the sample size gets bigger and bigger, your estimate $\widehat{\beta}$ will not converge to the true value, i.e. it is inconsistently estimated. Instead it converges to the true value plus some bias (which depends on the size of $\gamma$, the correlation between $x$ and $d$ and the variance of $d$).

So you see that OLS is not BLUE by definition as you describe it in point (1). It is only BLUE if it fulfills the conditions set by the Gauss-Markov theorem. Concerning point (2), if OLS satisfies these conditions, then it is a best linear predictor of the conditional expectation.