I have a large number of predictors that are hypothesized to be important in determining my binary outcome variable (here's a bit more about my goal, the predictors and the outcome). The problem is that considering the number of "positive cases" that I have, I have too many predictors (I have about 450 cases in total, about 100 positive cases, and more than 20 predictors.)
I'd like to use PCA to reduce my data, but many of my predictors are non-negative with a spike at zero and a long tail. For instance, the value for "the number of cigarettes per day smoked in the past 30 days" is zero for about 75% of the respondents and 99+ for a few.
I understand that there's nothing conceptually wrong with using PCA with such variables. But PCA finds the directions of maximum variance in data and I'm not sure that variance is the right measure of dispersion for this kind of data. (These variables also cause all sorts of errors and warnings to be generated when I use other data reduction software.)
I'd like to find a non-response-variable-driven way to deal with these variables. What I've tried already is to use $x \rightarrow log(x + 1)$ transformation followed by restricted cubic spline transforms (as implemented by $transcan$ function in Hmisc package in R), but they don't improve the situation much. I've seen this paper by Royston et al. that explicitly deals with this problem, but their approach is response-variable driven, which is something I'd like to avoid.
I'd love to hear everyone's thoughts, experiences, and suggestions.