Tell me more ×
Cross Validated is a question and answer site for statisticians, data analysts, data miners and data visualization experts. It's 100% free, no registration required.
up vote 25 down vote favorite
19

I have a dataset that has both continuous and categorical data. I am analyzing by using PCA and am wondering if it is fine to include the categorical variables as a part of the analysis. My understanding is that PCA can only be applied to continuous variables. Is that correct? If it cannot be used for categorical data, what alternatives exist for their analysis?

share|improve this question

3 Answers

up vote 24 down vote

Although a PCA applied on binary data would yield results comparable to those obtained from a Multiple Correspondence Analysis (factor scores and eigenvalues are linearly related), there are more appropriate techniques to deal with mixed data types, namely Multiple Factor Analysis for mixed data available in the FactoMineR R package (AFDM()). If your variables can be considered as structured subsets of descriptive attributes, then Multiple Factor Analysis (MFA()) is also an option.

The challenge with categorical variables is to find a suitable way to represent distances between variable categories and individuals in the factorial space. To overcome this problem, you can look for a non-linear transformation of each variable--whether it be nominal, ordinal, polynomial, or numerical--with optimal scaling. This is well explained in Gifi Methods for Optimal Scaling in R: The Package homals, and an implementation is available in the corresponding R package homals.

share|improve this answer
up vote 7 down vote

A Google search "pca for discrete variables" gives this nice overview by S. Kolenikov (@StasK) and G. Angeles. To add to chl answer, the PC analysis is really analysis of eigenvectors of covariance matrix. So the problem is how to calculate the "correct" covariance matrix. One of the approaches is to use polychoric correlation.

share|improve this answer
(+1) Thanks for the link.It is also possible to consider a heterogeneous correlation matrix (see, e.g. hetcor() from the polycor package). Provided the VC matrix is SDP, it should do the work--mostly in the spirit of Factor Analysis. Nominal variables might be dummy coded. – chl♦ Dec 28 '10 at 15:24
Thanks a lot for suggestions, these are great guides. – Nikolina Icitovic Dec 29 '10 at 15:25
Interesting -- this is a talk I prepared ages ago :). – StasK Aug 23 '11 at 4:50
@StasK, kudos :) It seems that not only me found this talk useful, otherwise it would not be on the top in gooogle search. This question pops up from time to time, so maybe you want to do a blog post about it for our community blog? – mpiktas Aug 23 '11 at 6:43
@StasK, I've edited the post to mention the authors of the overview. My initial intention was to demonstrate that searching on google can come up with good answers, so there is no explicit need to ask here. But this is not an excuse to not cite the authors, given the volatility of the internet. – mpiktas Aug 23 '11 at 6:48
show 1 more comment
up vote 0 down vote

I would suggest having a look at Non linear principal component analysis or categorical principal component analysis See Linting & Kooji 2012 Non linear principal component analysis with CATPCA: a tutorial, journal of personality assessment 94(1).

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.