3
$\begingroup$

It is clear that the first principal component is the vector which is the closest to the data, but can someone prove why the first two principal components span a plane that is the closest to the data?

$\endgroup$
2
  • $\begingroup$ I don't think closest to the data is correct. $\endgroup$
    – SmallChess
    Mar 10, 2017 at 14:17
  • 2
    $\begingroup$ check out the Mirsky-Eckart-Young theorem en.wikipedia.org/wiki/… $\endgroup$ Oct 18, 2022 at 17:06

1 Answer 1

1
$\begingroup$

The first PC (PC1) is the linear combination that maximizes variance. If you replace the data points with PC1, this is closest to the data in the sense that it minimizes the (Euclidean) norm of the residual. Now, PC2 maximizes variance among all linear combinations orthogonal to PC1.

If you replace the data points with (PC1, PC2) again, this is the plane closest to the point swarm in the sense of minimizing the Euclidean norm of the residual vector. See Geometric understanding of PCA in the subject (dual) space

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.