$S(\textbf{b})$ denotes the sum of squares of the model $$y=Z\beta+\sigma\epsilon$$ with usual assumptions.

Why is $Z'Z$ non negative definite? Does least square estimator still minimize sum of squares if the design matrix is not non-negative definite? If not, what should we do about it?

enter image description here


1 Answer 1


Design matrix $A$ can be any matrix, because we are getting the derivative of the orignal problem

$$\text{minimize}~~~ \|Ax-b\|^2$$

The derivative is


We set the derive to $0$, therefore we are solving the second equation, where $A^TA$ is symmetric positive semi-definite.


  • $\begingroup$ Why is that positive semidefinite? en.wikipedia.org/wiki/Positive-definite_matrix $\endgroup$
    – ZHU
    Commented Oct 11, 2016 at 3:25
  • $\begingroup$ I don't understand why $Z'Z$ is non negative definite? $\endgroup$
    – ZHU
    Commented Oct 11, 2016 at 3:30
  • $\begingroup$ So it is trivial as said? as a result of $x^2\geq 0$ $\endgroup$
    – ZHU
    Commented Oct 11, 2016 at 3:50
  • 2
    $\begingroup$ $x'Z'Zx=(Zx)'Zx\geq 0$ since each entry is a square of some real number? $\endgroup$
    – ZHU
    Commented Oct 11, 2016 at 3:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.