I'm trying to classify a variable into either 0 or 1, using 50 factors, with a sample size of 2000. 25% of the dependent variables are 0 and the rest are 1.
Of these factors, 30 are categorical. I've been having a lot of trouble doing PCA in R with the pcr function from the pls library:
--It seems that pcr will only allow a qualitative variable to have two categories. For example, my dependent variable can come from 5 countries. The pcr function will only allow my "country" factor to have 2 values--if there are 3, it throws an error. (Also annoying, the factor can only have one character--it will allow "A" but throw an error for "AFRICA")
--If I use model.matrix to get dummy variables for my qualitative factors, pcr starts going insane when the number of variables goes over 50 or so. It'll go from CV=.2717 to CV=3e8 from 49 comps to 50 comps, for example.
I'm wondering if PCA in general is not meant for high-dimensional data, particularly with a lot of categorical factors? Or am I just formatting my data incorrectly?
Note: I have used glmnet with alpha = 0, 1, and .5 (ridge, lasso, and elastic net regression) and it worked perfectly using the model.matrix. I wanted to compare my results with PCA, but I do know that there are alternatives for feature selection.