Would PCA work for boolean (binary) data types?

Question

I want to reduce the dimensionality of higher order systems and capture most of the covariance on a preferably 2 dimensional or 1 dimensional field. I understand this can be done via principal component analysis, and I have used PCA in many scenarios. However, I have never used it with boolean data types, and I was wondering if it is meaningful to do PCA with this set. So for example, pretend I have qualitative or descriptive metrics, and I assign a "1" if that metric is valid for that dimension, and a "0" if it is not (binary data). So for example, pretend you are trying to compare the Seven Dwarfs in Snow White. We have:

Doc, Dopey, Bashful, Grumpy, Sneezy, Sleepy and Happy, and you want to arrange them based on qualities, and did so as is:

$$\begin{pmatrix} & Lactose\ Intolerant & A \ Honor\ Roll & Athletic & Wealthy \\ Doc & 1 & 0 & 1 & 1 \\ Dopey & 0 & 0 & 0 & 0 \\ Bashful & 1 & 0 & 1 & 1 \\ Grumpy & 1 & 1 & 1 & 1 \\ Sneezy & 0 & 1 & 1 & 0 \\ Sleepy & 1 & 0 & 0 & 0 \\ Happy & 1 & 1 & 0 & 0 \end{pmatrix}$$

So for example Bashful is lactose intolerant and not on the A honor roll. This is a purely hypothetical matrix, and my real matrix will have many more descriptive columns. My question is, would it still be appropriate to do PCA on this matrix as a means of finding the similarity between individuals?

Answer 1

I would like to suggest you a relatively recent technique for automatic structure extraction from categorical variable data (this includes binary). The method is called CorEx from Greg van Steeg from University of Southern California. The idea is to use the notion of Total Correlation based on the entropy measures. It is appealing due to its simplicity and no tuning of large number of hyperparameters.

The paper about hierarchical representations (the most recent, builds on the top of the previous measures). http://arxiv.org/pdf/1410.7404.pdf

Answer 2

You can also use Multiple Correspondence Analysis (MCA), which is an extension of principal component analysis when the variables to be analyzed are categorical instead of quantitative (which is the case here with your binary variables). See for instance Husson et al. (2010), or Abdi and Valentin (2007). An excellent R package to perform MCA (and hierarchical clustering on PCs) is FactoMineR.

Answer 3

If you think of PCA as an exploratory technique to give you a way to visualise the relationships between variables (and in my opinion this is the only way to think about it) then yes, there is no reason why you can't put in binary variables. For example, here is a biplot of your data

It seems reasonably useful. For example, you can see that Doc and Bashful are very similar; that HR is rather unlike the three other variables; Sleepy and Sneezy are very dissimilar, etc.

Answer 4

Although PCA is often used for binary data, it is argued that PCA assumptions are not appropriate for binary or count data (see e.g. Collins 2002 for an explanation) and generalizations exists: the strategy is similar in spirit to the development of generalized linear models to perform regression analysis for data belonging to the exponential family.

An implementation in R of different methods can be found in the logisticPCA package, and a tutorial in this page.

Ref. Collins, M., Dasgupta, S., & Schapire, R. E. (2002). A generalization of principal components analysis to the exponential family. In Advances in neural information processing systems (pp. 617-624).