0
$\begingroup$

As the title/question implies I ran a PCA with mixed data, i.e. categorical and numeric, in R, once with the "FactoMineR" and "factoextra" packages (analysis version 1) and once with the "PCAmixdata" package (analysis version 2) (just to compare both approaches, out of curiosity).

The original dataset consists of 184 variables/953 samples and the idea behind the PCA was to reduce the dimensionality of the dataset in order to gain less but more informative components (variables). This all worked and after both analysis versions it seems that the first 5 components/dimension are sound (largest eigenvalues and visual inspection of the scree plot). Now I would like to use these 5 PCs as predictor variables in a subsequent (principal component) regression. And this is where I'm stuck. I don't know how to extract/export the PCs into new predictor variables so that each of the 953 subjects has a value in each of the 5 dimensions; I don't know which part of the output holds the information I need, or if further manual calculations will be necessary before I can extract the PC variables. If so, I wouldn't know the formula.

FYI: The output for both analysis versions produces eigenvalues of the dimensions, "coordinates", "Cos2" (-> quality of representation of a variable through a PC) and "contributions" (-> contributions (in percentage) of the variables to the principal components) for both the variables and individuals (samples). With the "PCAmixdata" package (analysis version 2) one can also examine the "squared loadings" and the "coefficents of the linear combinations defining the PC" for the variables only.

I am aware of the pcr() function of the "pls" package but it doesn't seem to work since it inherently performs a PCA first and regular PCAs - to my knowledge - cannot use categorical data.

Thank you for your answers in advance!

$\endgroup$
0
$\begingroup$

They are almost similar, if you run a pca using FactoMineR, you need the principal component score and they are stored under $svd$U because FactoMineR performs a pca using svd (you can see how they are related).

So for example using mtcars:

library(FactoMineR)
mpg = mtcars[,"mpg"]
pca.res = PCA(mtcars[,-1],graph=FALSE)

To get back the so called principal component scores, normally used in prcomp etc you do:

PCs = pca.res$svd$u %*% diag(pca.res$svd$d)

Because this is essentially multiplying the U matrix by its corresponding eigen value, for a regression this is not quite essential. However for complete sake, we can do:

PCs = PCs[,1:5]

Or you can do:

PCs = pca.res$ind$coord[,1:5]

followed by:

fit_PCs = lm(mpg ~ PCs)

If you use pcr() from pls:

fit_pls = pcr(mpg ~ .,data=mtcars,ncomp=5,
validation="none",scale=TRUE,center=TRUE)

The fitted values for 5 components are in:

fitted(fit_pls)[,,5]

So we check the fitted values from both:

plot(fitted(fit_pls)[,,5],fitted(fit_PCs),xlab="pls_5comp",ylab="PCA+lm")

enter image description here

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ Thank you SO much for the quick reply! This is extremely helpful, I could almost cry. Do I understand it correctly then that the principle component scores are - independent from what type of function I use - the scores I always have to look for and use when fitting the PCs as new variables in a subsequent regression? Do you know by any chance where the scores are stored when applying the PCAmixdata package? And if I may ask one last question concerning you comment on the pcr() function: Is it possible to use the function on mixed data in that case?? $\endgroup$ – daedhalus Apr 16 at 10:34
  • $\begingroup$ yes you look for the scores / components basically, to fit a new regression. I am not familiar with PCAmixdata, I think you need to find where this matrix is stored. Not so familiar with that package unfortunately $\endgroup$ – StupidWolf Apr 16 at 10:44
  • $\begingroup$ quickly looking at the package vignette it should be res.pcamix$ind$coord, cos it says ‘$coord’: factor coordinates (scores) of the individuals, again, i am not familiar with this so you should double check it $\endgroup$ – StupidWolf Apr 16 at 10:48
  • $\begingroup$ When I run res.pcamix$ind$coord for the pcamixdata package I get the same values as in running pca.res$ind$coord with FactoMineR. I suppose the values should however be identical to the output of pca.res$svd$U? $\endgroup$ – daedhalus Apr 16 at 12:03
  • $\begingroup$ Browsing the factoMineR FAQ side there is an entry which says: "Where do I find scores [and loadings] in res.pca? -> Scores (i.e. principal coordinates) are in: res.pca$ind$coord The variance of the individuals' coordinates for a dimension corresponds to the eigenvalue of this dimension." Now I am just more confused about whether the scores are in scd§U or in ind§coord? $\endgroup$ – daedhalus Apr 16 at 12:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.