Mostly Harmless Econometrics, explanation of solution to the population least squares problem

Question

I'm trying to make sense of Mostly Harmless Econometrics's explanation of the solution to the population least squares problem, but I'm not following Angrist and Pischke's argument. Here's what they write (on pg. 35):

This section is concerned with the vector of population regression coefficients, defined as the solution to a population least squares problem. At this point we are not worried about causality. Rather, we let the Kx1 regression coefficient vector $\beta$ be defined by solving $\beta = argmin_{b} E[(Y_{i} - X'_{i}b)^2]$. Using the first-order condition, $E[X_i(Y_i - X_i'b)] = 0$, the solution can be written $\beta = E(X_iX'_i)^{-1}E(X_iY_i)$. Note that by construction, $E(X_i(Y_i - X'_i\beta)) = 0$. In other words, the population residual, which we defined as $Y_i - X'_i\beta = e_i$, is uncorrelated with the regressors, $X_i$. It bears emphasizing that this error term does not have a life of its own. It owes its existence and meaning to $\beta$.

What I don't get is the sentence that starts "using the first-order condition." Why are we allowed to assume that first-order condition holds here?

I should mention that at this point in the book, Angrist and Pischke have already proved that a random variable $Y_i$ may be decomposed as $Y_i = E(Y_i | X_i) + \epsilon_i$, where $\epsilon_i$ is uncorrelated with any function of $X_i$. But I don't see how they can apply that theorem here.

Am I missing something simple, or is the book's explanation just confusing?

Answer 1

"First-order condition" in optimization just means taking the first derivative and setting it to zero. The second derivative here is clearly negative, so under mild regularity conditions the first-order condition will give you the optimal solution. You don't need to "assume" that equation--it's just something you want to solve that you know will give you an optimum under very general conditions. The population linear regression problem is by definition the solution to that equation.

Taking the first-order equation to be true directly implies that the residuals aren't correlated with $X$. You know the residuals have expectation zero, so $Cov(X,\epsilon) = E[X\epsilon] - E[X] E[\epsilon] = E[X\epsilon] = 0$ when you are solving the first-order condition.

I honestly agree with you that Mostly Harmless is confusing here and elsewhere. I'm a statistician, though, so I often find economists' ways of presenting stats frustrating ;P