0
$\begingroup$

I'm working with morphometric data including 50 different variables (with very different scales) measured in 180 individuals (these 180 individuals belong to 4 different groups which received different treatments) and I'm trying to analyze these data by PCA in order to identify the variables, or combinaisons of variables, which are the most useful to discriminate these individuals.

If I understand things correctly, Z-Score is classically used to standardize data before PCA analysis in order to obtain variables with a mean = 0 and SD = 1. However, most of my data are not normally distributed. This will thus affect the "quality" of the standardization since Z-Scores are based on the mean and SD of the data. Therefore, I'm trying to use Robust Z-Scores (based on median and MAD) instead of the classical Z-Scores. After standardization by the Robust Z-Scores, all my variables have a median = 0 and a MAD = 0.67.

I'd like to know if the standardization based on Robust Z-Scores (Median & MAD) instead of classical Z-Scores (Mean & SD) is suitable for PCA analysis.

Thanks a lot for your precious help !

$\endgroup$
3
  • $\begingroup$ The use of Z-scores for standardization means your PCA is based on correlations rather than covariances. There is no requirement in most PCA applications that the distribution by Normal and there is no requirement, period, for Z-scores to be useful and meaningful, apart from the need for the SD to be nonzero so that the Z-score can be defined. $\endgroup$
    – whuber
    Dec 2, 2021 at 13:52
  • $\begingroup$ How are you intending to use the model? Will you be applying it to predictions on new data? If so, how will you apply the "robust z-score" transformation to the new data? $\endgroup$
    – EdM
    Dec 2, 2021 at 14:07
  • 1
    $\begingroup$ The PCA will not be used for predictions on new data but to identify the variables, or combinaisons of variables, which are the most useful to discriminate the individuals (the 180 individuals belong to 4 groups which received different treatments). $\endgroup$
    – Erik
    Dec 3, 2021 at 16:33

1 Answer 1

1
$\begingroup$

PCA first finds the linear combination of variables that account for the most variance, then the orthogonal combination that accounts for the next most variance, and so on. It does so in whatever scales the variables are presented, after any transformations that you might have previously performed. So the question to ask with respect to pre-processing for PCA is what transformation will best maintain comparability of scales among the various predictors. That's hard to say without knowing more about the nature of the data.

One thing to think about is what the variances within each variable will be after transformation, because PCA will be based on the post-processed variances regardless of how you pre-processed the data.

Consider a variable with a lot of outlying extreme values. It will tend to have a very high variance in its initial scale, but unless more than half the values are extreme the MAD might be fairly small. Such a variable will thus have its range reduced more by the "classical Z-score" (division by standard deviation) than by the "robust Z-score" (division by MAD). Its post-processing variance would then be higher following a "robust Z-score" than "classical Z-score" pre-processing. Is that what you want? Will that be the best way to maintain comparability among variables after transformation? I'm afraid that only you and your colleagues can answer that, based on your understanding of the subject matter.

The above assumes that your variables are continuous. If any of your variables are binary or categorical, you have additional issues to consider in trying to make the scales comparable, outlined for example here.

Finally, I fear that PCA might not be the best way "to identify the variables, or combinaisons of variables, which are the most useful to discriminate these individuals" as you wish. Each principal component might well contain contributions from all your variables. That reduces the dimensionality of the problem without reducing the number of original variables. That said, if you were to use a method like LASSO or elastic net you would face the same choices about pre-processing as you do for PCA. Consider whether tree-based or other modeling approaches, some of which are less sensitive to relative scales among predictor variables, might better help you reach your stated goal.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.