How I can deal with too many variables in training a data set? I am trying to train a predictive model on whether a given person is ( male or female) based on behavior cues we've obtained from online surveys.  


*

*The dependant variable will be a binary ( 1 or 0 ).  

*For the training data set, I have a list of 150+ numeric variables ranging from issues as far apart as "political leanings" to "amount daily exercise".  These are all indexed numbers and range from 1 to 100. 
So my problem is, I have too many variables!
I don't really want to assume some variables are more important than others because there may be something I don't know about. And I don't want to throw away important correlations between variables. But how can I build a good, strong, predictive model using all the data I have? 
 A: If you are only interested in predictive performance, then simply using regularisation (e.g. ridge regression, SVM, regularised logistic regression etc.) will probably be as good as anything.  Feature selection is difficult as it is easy to over-fit whatever criterion you use to select the features (as there are so many degrees of freedom involved), whereas for regularisation, you generally only have to optimise a small number (often only one) regularisation parameters.  This is pretty much the advice given in the appendix of Miller's book on "subset selection in regression".  The LASSO type regularisation results in feature selection, but that doesn't mean it necessarily gives better predictive performance than the standard L2 "ridge" regression approach.
A: Personally, I think it's a mistake to presume that ALL of the variables can contribute in some reasonable way to the target variable; surely some are co-correlated, and some are truly unrelated to your target variable, so these issues will confound the analysis.
If you're determined to try using all, using a method like the LASSO to weight the contributions of variables would be a possible solution. The theory of the LASSO method is that it will dampen out the negative contributions of less-related variables by using penalty terms.
The LASSO regression method is implemented in R (pdf), so at least there's no "cost" barrier to trying it.
Another alternative would be to use Principal Component Analysis to "compress" the information contained in subsets of variables into a smaller number of representative variables.
