5
$\begingroup$

$S(\textbf{b})$ denotes the sum of squares of the model $$y=Z\beta+\sigma\epsilon$$ with usual assumptions.

Why is $Z'Z$ non negative definite? Does least square estimator still minimize sum of squares if the design matrix is not non-negative definite? If not, what should we do about it?

enter image description here

$\endgroup$
0

1 Answer 1

4
$\begingroup$

Design matrix $A$ can be any matrix, because we are getting the derivative of the orignal problem

$$\text{minimize}~~~ \|Ax-b\|^2$$

The derivative is

$$2(A^TAx-A^Tb)$$

We set the derive to $0$, therefore we are solving the second equation, where $A^TA$ is symmetric positive semi-definite.

https://math.stackexchange.com/questions/133350/help-me-understand-a-line-in-an-ata-is-positive-semi-definite-proof

$\endgroup$
4
  • $\begingroup$ Why is that positive semidefinite? en.wikipedia.org/wiki/Positive-definite_matrix $\endgroup$
    – ZHU
    Oct 11, 2016 at 3:25
  • $\begingroup$ I don't understand why $Z'Z$ is non negative definite? $\endgroup$
    – ZHU
    Oct 11, 2016 at 3:30
  • $\begingroup$ So it is trivial as said? as a result of $x^2\geq 0$ $\endgroup$
    – ZHU
    Oct 11, 2016 at 3:50
  • 2
    $\begingroup$ $x'Z'Zx=(Zx)'Zx\geq 0$ since each entry is a square of some real number? $\endgroup$
    – ZHU
    Oct 11, 2016 at 3:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.