# Does least square estimator still minimize sum of squares if the moment matrix is not non-negative definite?

$S(\textbf{b})$ denotes the sum of squares of the model $$y=Z\beta+\sigma\epsilon$$ with usual assumptions.

Why is $Z'Z$ non negative definite? Does least square estimator still minimize sum of squares if the design matrix is not non-negative definite? If not, what should we do about it?

Design matrix $A$ can be any matrix, because we are getting the derivative of the orignal problem

$$\text{minimize}~~~ \|Ax-b\|^2$$

The derivative is

$$2(A^TAx-A^Tb)$$

We set the derive to $0$, therefore we are solving the second equation, where $A^TA$ is symmetric positive semi-definite.

https://math.stackexchange.com/questions/133350/help-me-understand-a-line-in-an-ata-is-positive-semi-definite-proof

• Why is that positive semidefinite? en.wikipedia.org/wiki/Positive-definite_matrix
– ZHU
Oct 11, 2016 at 3:25
• I don't understand why $Z'Z$ is non negative definite?
– ZHU
Oct 11, 2016 at 3:30
• So it is trivial as said? as a result of $x^2\geq 0$
– ZHU
Oct 11, 2016 at 3:50
• $x'Z'Zx=(Zx)'Zx\geq 0$ since each entry is a square of some real number?
– ZHU
Oct 11, 2016 at 3:55