I am currently using principal components analysis to select variables to use in modelling. At the moment, I make measurements A, B and C in my experiments -- What I really want to know is: Can I make fewer measurements and stop recording C and or B to save time and effort?
I find that all 3 variables load heavily onto my first principal component which accounts for 60% of the variance in my data. The component scores tell me that if I add these variables together in a certain ratio (aA+bB+cC). I can get a score on PC1 for each case in my dataset and could use this score as a variable in modelling, but that doesn't allow me to stop measuring B and C.
If I square the loadings of A and B and C on PC1, I find that variable A accounts for 65% of the variance in PC1 and variable B accounts for 50% of the the variance in PC1 and variable C also 50%, i.e. some of the variance in PC1 accounted for by each variable A, B and C is shared with another variable, but A comes out on top accounting for slightly more.
Is it wrong to think that I could just choose variable A or possibly (aA+bB, if necessary) to use in modelling because this variable describes a large proportion of the variance in PC1 and this in turn describes a large proportion of the variance in the data?
Which approach have you gone for in the past?
- Single variable which loads heaviest on PC1 even if there are other heavy loaders?
- Component score on PC1 using all variables even if they are all heavy loaders?