I am trying to do a Principal Component Analysis to reduce my data from a test battery. I have a small sample N=58 (It is a selected kind of dog population and due to the Pandemics I am unable to continue my data collection) I originally had 98 variables from my test, I did a pre-data exclusion by let out variables with low variation (<20%) and those highly correlated (taking forward the most relevant of the pair). I finally got 49 variables. However, when I run the PCA in SPSS, the KMO is very low as well as the determinant of the correlations matrix. I already checked the correlation matrix from the PCA for variables not showing correlations >.3 and those highly correlated <.7 (I found 1). Therefore, do you have any advice about how can I take out more variables in an objective, valid way, so I can increase the adequacy of my PCA? Thanks a lot
2 Answers
A couple of points here:
The Kaiser-Meyer-Olkin index test is used before factor analysis not PCA. While these two analyses are similar, they are not the same. This has been covered in many posts, but for the sake of this post I'll just note that FA is performed on the covariance matrix, while PCA is performed directly on the data.
That being said, the KMO is telling us that you have a very low sample size. Additionally, PCA is generally unstable an inadvisable when your sample size is below about 200-300. You can check this by bootstrapping your PCA. You will most likely see that you get widely different PCs depending on the bootstrap iteration.
As far as how you should move forward, instead of using dimensionality reduction, I would suggest some sort of variable selection method when you model your data (assuming that is what you plan to do). A random forest model may be useful considering your sample size-to-variables ratio.
-
2$\begingroup$ Could you elaborate on what you mean by "unstable" and "inadvisable" in (2)? After all, there are a great many published, peer-reviewed applications of PCA with much smaller datasets than 200 observations. Your rule of thumb must depend on how many variables you have, on how they were obtained and measured, on the objectives of the PCA, on which PCA outputs you are specifically concerned about, and even on whether you are appropriately comparing multiple outputs during the bootstrapping (it's hard to compare collections of subspaces in a simple way). $\endgroup$– whuber ♦Jul 7, 2021 at 18:29
I would analyze the covariance matrix of your data to see whether it is different from the random matrix, e.g. you could apply Marchenko-Pastur theorem to the eigen values spectrum. If you determine that all eigenvalues are withim the bounds of the random matrix, then not much can be done with PCA to reduce dimensionality.
