$S(\textbf{b})$ denotes the sum of squares of the model $$y=Z\beta+\sigma\epsilon$$ with usual assumptions.

Why is $Z'Z$ non negative definite? Does least square estimator still minimize sum of squares if the design matrix is not non-negative definite? If not, what should we do about it?

enter image description here


Design matrix $A$ can be any matrix, because we are getting the derivative of the orignal problem

$$\text{minimize}~~~ \|Ax-b\|^2$$

The derivative is


We set the derive to $0$, therefore we are solving the second equation, where $A^TA$ is symmetric positive semi-definite.


  • $\begingroup$ Why is that positive semidefinite? en.wikipedia.org/wiki/Positive-definite_matrix $\endgroup$ – ZHU Oct 11 '16 at 3:25
  • $\begingroup$ I don't understand why $Z'Z$ is non negative definite? $\endgroup$ – ZHU Oct 11 '16 at 3:30
  • $\begingroup$ So it is trivial as said? as a result of $x^2\geq 0$ $\endgroup$ – ZHU Oct 11 '16 at 3:50
  • 2
    $\begingroup$ $x'Z'Zx=(Zx)'Zx\geq 0$ since each entry is a square of some real number? $\endgroup$ – ZHU Oct 11 '16 at 3:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.