Creating new variables from PCA loadings

Question

I'm (very) new to PCA and confused about how to use the output of a PCA analysis to construct new variables that will be used as predictors in a regression analysis. I've looked at previous questions (e.g. Creating a new PC variable based on PCA loadings) but I'm still not sure I'm doing the right thing, as my intent is that the PCA loadings will be used to weight a new variable, rather than one that currently exists (as is the case in the existing question).

I've conducted a PCA analysis using the principal function in the R psych package. A Scree plot suggested 2 components was sensible.

twopca <- principal(facto, nfactors=2, rotate="varimax")

The loadings output looks like this:

Loadings:
            RC1    RC2   
articulate          0.826
ambitious           0.557
competent           0.872
hardworking         0.316
intelligent         0.848
reliable            0.734
cheerful     0.831       
downtoearth  0.448       
fun          0.709       
friendly     0.863       
pleasant     0.845       
warm         0.873       

             RC1   RC2
SS loadings    3.735 3.127
Proportion Var 0.311 0.261
Cumulative Var 0.311 0.572

I have two questions. First, what exactly are these loadings? Second, if I wanted to create two new variables based on these loading weights, is it possible do something like:

newvar1 = (cheerful * 0.831) + (downtoearth * 0.448) .... + (warm * 0.873)
newvar2 = (articulate * 0.826) + ... (reliable * 0.734)

This has been suggested to me, but I want to make sure it's legit (and part of that comes from understanding what these loadings mean in the first place). Thankyou for any help !

Answer 1

It seems that the package psych dos some nonstandard things, which are not always necessary for PCA regression. I suggest you use a different method. A straightforward one is to use the function prcomp that comes with the default stats package or do PCA by simply taking the singulr value decompositon (svd) of the data matrix appropriately standardised.

Minimal example:

set.seed(55)
X = matrix(rnorm(100), 20)
X = scale(X) #centres the vriables and scales them to unit variance

pca_stats = prcomp(x = X) #from the stats package

pca_svd = svd(X) # the svd of X

# the loadings are the same
pca_svd$v
pca_stats$rotation

#one needs to compute the scores (the "new" variables, i.e. the PCs' scores) by hand for the prcomp output
pca_stats$scores = X %*% pca_stats$rotation

# one may want to rescale the left singular value to obtain standard PCs as
pca_svd$u = pca_svd$u %*% diag(pca_svd$d)

now you have two sets of identical PCs

sum(abs(pca_stats$scores - pca_svd$u))

and you can select the first two PCs with

PCs_stats = pca_stats$scores[, 1:2]
# or
PCs_svd = pca_svd$u[, 1:2]

Note that here I have assumed that you want to run PCA on the correlation matrix as opposed to the covariance matrix. This is most likely to be the best approach.

The package psych by default rotates the PCs. This is not always accepted and may cause the rotated components to be correlated (depending on how the loadings are scaled before rotation). However, since PCA is carried out not considering the response variables, it is not assured that the standard PCs will be better predictors than the rotated ones. So you may want to try out some rotatons as well, for example with

pca_svd_vari = varimax(pca_svd$v[, 1:2], normalize = FALSE) # or TRUE
PCs_svdRot = X %*%  pca_svd_vari$loadings

There are several details I did not discuss. Maybe you should do some reading before using PCA regression?

Answer 2

First, every variable should have a loading on every factor. The output is made this way for ease of reading. I'm not familiar with the psych package and questions about code are off-topic here.

Second, yes, you are doing the right thing, but you need all the loadings.

Third, R doubtless lets you capture this without doing it by "hand". But, again, questions about code are off topic here.