I'm trying to classify a variable into either 0 or 1, using 50 factors, with a sample size of 2000. 25% of the dependent variables are 0 and the rest are 1.

Of these factors, 30 are categorical. I've been having a lot of trouble doing PCA in R with the pcr function from the pls library:

--It seems that pcr will only allow a qualitative variable to have two categories. For example, my dependent variable can come from 5 countries. The pcr function will only allow my "country" factor to have 2 values--if there are 3, it throws an error. (Also annoying, the factor can only have one character--it will allow "A" but throw an error for "AFRICA")

--If I use model.matrix to get dummy variables for my qualitative factors, pcr starts going insane when the number of variables goes over 50 or so. It'll go from CV=.2717 to CV=3e8 from 49 comps to 50 comps, for example.

I'm wondering if PCA in general is not meant for high-dimensional data, particularly with a lot of categorical factors? Or am I just formatting my data incorrectly?

Note: I have used glmnet with alpha = 0, 1, and .5 (ridge, lasso, and elastic net regression) and it worked perfectly using the model.matrix. I wanted to compare my results with PCA, but I do know that there are alternatives for feature selection.

  • 1
    $\begingroup$ A factor is not a number at all--it just indicates a category. With two factors there is only one way to assign numbers to the factors (up to location and scale, both of which are normalized by PCA), but with more than two there is no unique way. What are you hoping PCA with factors (whatever that might possibly mean) is going to do for you, then? $\endgroup$ – whuber Feb 9 '15 at 22:22
  • $\begingroup$ @whuber I was going off of the statistical learning book "An Introduction to Statistical Learning" by Gareth James. He has a dependent variable that is the salaries of baseball players, and several factors that are things like "League" etc. It seems like it just so happens that the qualitative factors are binary--i.e. "major" and "minor." So they work with PCA. But what if there was a third league? I'm not sure what would happen. $\endgroup$ – Nagy Feb 10 '15 at 2:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.