I want to do principal component analysis (factor analysis) on SPSS based on 22 variables. However, some of my variables are very skewed (skewness calculated from SPSS ranges from 2–80!).

So here are my questions:

  1. Should I keep the skewed variables like that or could I transform the variables on principal component analysis? If yes, how would I interpret factor scores?

  2. What type of transformation should I do? log10 or ln?

  3. Originally, my KMO (Kaiser–Meyer–Olkin) is 0.413. Much literature recommends a minimum of 0.5. Can I still do factor analysis, or do I need to remove variables to raise my KMO to 0.5?

  • 7
    $\begingroup$ One note: PCA is not the same as factor analysis. PCA is a data reduction method, FA is an attempt to find latent variables. They often (but not always) give similar results $\endgroup$
    – Peter Flom
    Oct 5, 2012 at 10:24

2 Answers 2

  1. Skewness issue in PCA is the same as in regression: the longer tail, if it is really long relative to the whole range of the distribution, actually behaves like a big outlier—it pulls the fit line (principal component in your case) strongly toward itself because its influence is enhanced; its influence is enhanced because it is so far from the mean. In the context of PCA allowing very skewed variables is pretty similar to doing PCA without centering the data (i.e., doing PCA on the basis of cosine matrix rather than correlation matrix). It is you who decides whether to permit the long tail to influence results so greatly (and let the data be) or not (and transform the data). The issue is not connected with how you do interpretation of loadings.

  2. As you like.

  3. KMO is an index that tells you whether partial correlations are reasonably small to submit data to factor analysis. Because in factor analysis we generally expect a factor to load more than just two variables. Your KMO is low enough. You can make it better if you drop from the analysis variables with low individual KMO values (these form the diagonal of anti-image matrix, you can request to show this matrix in SPSS Factor procedure). Can tranforming variables into less skewed recover KMO? Who knows. Maybe. Note that KMO is important mostly in Factor analysis model, not Principal Components analysis model: in FA you fit pairwise correlations, whereas in PCA you don't.

  • $\begingroup$ Thanks a lot! Can you advice me on citable literature on why KMO makes sense for FA but not for PCA (3.)? $\endgroup$ Dec 28, 2022 at 9:57
  • 1
    $\begingroup$ @Nicole you may use KMO as one of the tools to select variables before PCA if you are going to use PCA as a method of factor extraction. However, PCA is not quite suited or always suited to perform factor analytic task, true FA is better for this. Any textbook on FA should say what I've just said. $\endgroup$
    – ttnphns
    Dec 28, 2022 at 10:12
  • $\begingroup$ Thank you. I haven't done FA for a long time. As for PCA I have never heared to use KMO until my reviewer asked me why I didn't do it. Now I'll try to back up my answer with references. $\endgroup$ Dec 28, 2022 at 12:08
  • 1
    $\begingroup$ @Nicole there are assumptions of FA (even if you are using PCA as a method of FA). There KMO is mentioned. $\endgroup$
    – ttnphns
    Dec 28, 2022 at 13:21
  • $\begingroup$ Wow, splendid! Thank you :) $\endgroup$ Dec 28, 2022 at 13:44

+1 to @ttnphns, I just want to expand a little on point #2. Transformations are often used to stabilize skew. As @ttnphns points out, you would use these before you run your analyses. Log transformations are part of the Box-Cox family of power transformations. You will want to consider a wider range of possible transformations than just logs (e.g., square root, reciprocal, etc.). The choice between different logarithmic bases has no effect on the strength of the transformation. When people are going to work mathematically with the transformed variable, natural logs are sometimes preferred, as the natural log can make for cleaner math in some cases. If you don't care about that, you may want to pick a base that will facilitate interpretation. That is, each unit increase in the new scale will represent a base-fold increase in the original scale (e.g., if you used log base 2, then every unit would be a 2-fold increase, base 10 means every unit would be a 10-fold increase, etc.), so it can be nice to pick a base that such that your data will span several units in the transformed scale.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.