I was reading about the Principal Component Analysis algorithm. I don't understand why, in order to do dimensionality reduction, we create the covariance matrix and then we extract its eigenvectors.
Compute "covariance matrix":
$$ \Sigma = \frac 1 m \sum_{i=1}^n(x^{(i)})(x^{i)})^T $$ Compute "eigenvectors" of matrix $\Sigma$:
$$ \color{darkblue}{\texttt{[U, S, V]} = \texttt{svd(Sigma);}} $$
After that, why do we select the first k eigenvectors? Why don't we do some ranking of groups of k eigenvectors and then select the best group?