What are good metrics to assess the quality of a PCA fit, in order to select the number of components? What is a good metric for assessing the quality of principal component analysis (PCA)?
I performed this algorithm on a dataset. My objective was to reduce the number of features (the information was very redundant). I know the percentage of variance kept is a good indicator of how much information we keep, be are there other information metrics I can use to make sure I removed redundant information and didn't 'lose' such information?
 A: There are also measures based on information-theoretic criteria like
Rissanen's MDL (and variations)
A: I assume part of this question is whether other metric exist besides the cumulative percent variance (CPV) and the similar scree plot approach. The answer to this is, yes, many.
A great paper on some options is Valle 1999:

*

*Selection of the Number of Principal Components:  The Variance of the Reconstruction Error Criterion with a Comparison to Other Methods
Sergio Valle, Weihua Li, and S. Joe Qin, Industrial & Engineering Chemistry Research 1999 38 (11), 4389-4401
It goes over CPV, but also Parallel Analysis, Cross-validation, Variance of the reconstruction error (VRE), information criteria based methods, and more. You might follow the recommendation made by the paper after comparing and use the VRE, but cross-validation based on PRESS also works well in my experience and they get good results with that too. In my experience, CPV is convenient and easy, and does a decent job, but those two methods are usually better.
There are other ways to evaluate how good your PCA model is if you know more about the data. One way is to compare the estimated PCA loadings to the true ones if you know them (which you would in simulations). This can be done by calculating the bias of the estimated loadings to the true ones. The bigger your bias, the worse your model. For how to do that, you can check out this paper where they use this approach to compare methods. It is not usable in real data cases though, where you don't know the true PCA loadings. This speaks less to how many components you removed, than to the bias of your model due to the influence of outlying observations, but it still serves as a model quality metric.
