# Principle Component Analysis on categorical predictors [duplicate]

I tried running prcomp() on my training set, which contains some categorical/factor predictors (as well as a binary response), and was given an error saying my data needs to be numeric. Can PCA not be performed at all with categorical features? Or is there a way to work around the function...

Thank you!

## marked as duplicate by mkt, Peter Flom♦Aug 2 at 11:47

• – mkt Aug 2 at 11:13

Take my answer as a comment more than a true answer (I am a new contributor so i cannot comment yet). If you can compute the varcov of the variables, then you can use PCA on that varcov matrix: of course you can compute the covariances between random variables even when they are binomial variables that numerically represent a categoriacal variable referred to two categories only (the same holds for multi-category variables). So be sure that you are representing your categorical variable via numbers (0-1 for a binomial categoriacal variable or 0-1-2,... for a categorical variable with more than 2 categories) and calculating the varcov correctly. Having said that, personally, I would prefer to keep them outside the PCA, especially if they are binomial and especially if you have just a few of them compared to the total number of features: for example, transform the set of non-categorical features via PCA to obtain a set of orthogonal features, then add the categorical variables to the set of simplified orthogonal features.

Code in python for the spectral decomposition of a simple varcov matrix obtained from two binomial variables

import numpy as np
import scipy.linalg as la
a=[0,1,0,1,0,1]
b=[0,0,0,0,1,1]
eigval, eigvec=la.eig(np.cov(a,b))


The short answer is yes, PCA is intended for continuously scaled features. Categorical features which are nominally scaled (e.g., the set of teams in the NFL is a nominally scaled variable) and stored as such would cause this function to give an error.

One workaround would be to impose a ranking on a categorical feature that would convert it into an acceptable ordinal scale but that would be an imposition of a rule.

Another workaround would be to convert the categorical features into a set of dummy variables or numeric effects as described in the link below. There are those that contend that PCA is invariant to mixtures of numeric variables such as dummy, ordinal and higher scaled features but that has not been my experience. My experience is that you get results with imbalanced latent dimensions which, based on the loadings, are heavily skewed towards the dummy variables.

My opinion is that the best solution would be to use correspondence analysis (CA). CA is described as 'PCA for categorical features'. It is well reviewed in the second link below. It may require grouping the continuous variables with a large range of possible values into a smaller set of buckets, depending on the software used.