Inverse covariance matrix vs covariance matrix in PCA In PCA, does it make a difference if we pick principal components of the inverse covariance matrix OR if we drop eigenvectors of the covariance matrix corresponding to large eigenvalues?
This is related to the discussion in in this post.
 A: Observe that for positive definite covariance matrix $\mathbf \Sigma = \mathbf{UDU}'$ the precision is $\boldsymbol \Sigma^{-1} = \mathbf {U D}^{-1} \mathbf U'$. 
So the eigenvectors stay the same, but the eigenvalues of the precision are the reciprocals of the eigenvalues of the covariance. That means the biggest eigenvalues of the covariance will be the smallest eigenvalues of the precision.  As you have the inverse, positive definiteness guarantees all eigenvalues are greater than zero. 
Hence if you retain the eigenvectors relating to the $k$ smallest eigenvalues of the precision this corresponds to ordinary PCA. Since we have already taken reciprocals ($\bf D^{-1}$), only the square root of the precision eigenvalues should be used in order to complete the whitening of the transformed data. 
A: In addition, the inverse covariance matrix is proportionnal to the partial correlation between the vectors:
Corr(Xi, Xj | (Xothers )

Correlation between Xi and Xj when all others are fixed, it is very useful for time series.
