# Least Square Estimation using Matrix

In the paragraph shown here (from "Least Squares Estimation" by S.A. Geer) I don't understand how we convert the least square estimation problem into the equation 4.

• You have to find such $\hat{\beta}$ that minimizes the square error so equation 4 is obtained by taking derivative $$\frac{\partial}{\partial \beta}\Vert Y - X\beta \Vert^2 = 0$$ Commented Feb 19, 2017 at 8:43
• I would be surprised if this question weren't answered in the passage immediately following the one shown.
– whuber
Commented Feb 19, 2017 at 18:19

An informal derivation : your model is

$$\mathbf y = \mathbf X \beta +\mathbf u$$

premultiply by $\mathbf X'$ to get

$$\mathbf X'\mathbf y = \mathbf X'\mathbf X \beta +\mathbf X'\mathbf u$$

Now pre-multiply by $(\mathbf X'\mathbf X)^{-1}$ to get

$$(\mathbf X'\mathbf X)^{-1}\mathbf X'\mathbf y = (\mathbf X'\mathbf X)^{-1}\mathbf X'\mathbf X \beta +(\mathbf X'\mathbf X)^{-1}\mathbf X'\mathbf u$$

The error term is unknown so ignore the last term, and simplify to get

$$(\mathbf X'\mathbf X)^{-1}\mathbf X'\mathbf y = \hat \beta$$

and it is only estimated and not the exact $\beta$, because we have ignored the term $(\mathbf X'\mathbf X)^{-1}\mathbf X'\mathbf u$.

The formal treatment that validates the optimality of this approach under a certain criterion, is what a comment suggested.

• Than you very much.
– wmac
Commented Feb 20, 2017 at 3:26