I have 8 variables that I want to combine into one performance score. I am doing PCA and taking the first three components (~62% of variance), then adding the three scores for each record from these components to get one final score. My question: four variables load high on the first component, three load high on the second component, and one variable loads high on the third component. Does that mean that the last variable that is loading high on the third component will have more impact on the final score than the other variables? I am asking since I want equal representation for all variables in my final score. Thank you for your help in advance.

  • $\begingroup$ Have you considered not doing PCA? Given that you don't have a lot of variables, I don't see why you want to do it at the first place. $\endgroup$ – jeff Jun 10 '16 at 4:20
  • $\begingroup$ In order to see directly how much "impact" a variable has on a component score you should consider component score coefficients, not loadings. $\endgroup$ – ttnphns Jun 10 '16 at 7:31
  • $\begingroup$ @HalilPazarlama what do you recommend an alternative to PCA? I have eight variables, some are very highly correlated to each other like customer satisfaction and resolution rate, and some are not correlated with the rest, and I want to combine all these important measures into one performance score. $\endgroup$ – Tarek Soukieh Jun 10 '16 at 23:39
  • $\begingroup$ @TarekSoukieh I would at least try to measure the performance without PCA, to see what PCA is improving. And well, if there is a high correlation, PCA is actually decorrelating the feature so it's guaranteed that you won't have equal impact from all. $\endgroup$ – jeff Jun 11 '16 at 12:17
  • $\begingroup$ @HalilPazarlama: that makes sense, I will try to do it both ways to see what PCA is improving. I have created a new question with more clarity on this, I appreciate it if you can take a look, thanks: stats.stackexchange.com/questions/218709/… $\endgroup$ – Tarek Soukieh Jun 13 '16 at 18:02

Yes, it probably does mean that the last variable will have a higher impact (I write "probably", as it depends on the magnitude of the the loadings and the standard deviation of the original variables).

The way to achieve your requirement of "equal representation for all variables in my final score" is most readily met by standardizing each of the variables (e.g., dividing each by its standard deviation) and then summing them. But, by doing PCA you are implicitly saying that you do not want to have each variable have the same impact, and are instead trying to ensure that each of three dimensions in the underlying dimensional space are equally represented.

To add some more complication: a three component solution only explaining 62% of the variance of 8 variables is not a strong fit, so some of your loadings are probably pretty poor, making the whole analysis a but suspect.

  • $\begingroup$ Thank you very much, let me clarify more. I have total 8 variables, 2 variables from each support channel, and I want all channels to be represented equally, but I thought about PCA to combine variables that are highly correlated (e.g. customer satisfaction and resolution rate), and these 8 variables have enough loadings on the first three components. Do I standardize them by std dev and add them into one score or do I combine them in PCA? $\endgroup$ – Tarek Soukieh Jun 10 '16 at 23:54
  • $\begingroup$ I would need a lot more information in order to answer this. I would suggest you try and ask a new question, but spend quite a bit of time working out precisely what it is you want to know. $\endgroup$ – Tim Jun 11 '16 at 12:19
  • $\begingroup$ I have created a new question, I would appreciate it if you can answer it, thank you. stats.stackexchange.com/questions/218709/… $\endgroup$ – Tarek Soukieh Jun 13 '16 at 17:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.