Exploratory data analysis for a dataset with continuous and categorical variables I have a data set which has DV and around 40 IVs. I want to select best variables out of the existing ones. I can use correlation, but it requires only numeric variables. I would like to see relation between continuous and categorical variables.
What method should I use for variable selection, which can handle both continuous and categorical variables (including relation between them)? I am using R as a modeling tool. Also, is it advisable to convert continuous variables into categorical variables for better results?
 A: First of all, it is possible to calculate correlation for both continuous and categorical variables, as long as the latter ones are ordered. This type of correlation is referred to as polychoric correlation.
In order to calculate polychoric correlation, since you plan to use R, you have, at least, two options: 1) psych package offers polychoric() and related functions (http://www.personality-project.org/r/psych/help/tetrachor.html); 2) package polycor offers hetcor() function. Analysis of models, containing ordered categorical (ordinal) variables, include some other methods, including, but not limited to, numeric recoding, ordinal regression and latent variables approach.
A: What you are looking for is multiple regression. Use the lm() function in R to create your models. Without context, it is hard to help you further. For instance, what does "best" variables mean? Something that significantly predicts something else doesn't mean it is interesting. Being male is highly predictive having a penis, but I don't know why you'd want to show that. If you don't care about interpreting coefficients and you only care about having the best predictions for y, then use all the variables. It will give you the highest $R^2$ value. If you are testing variables at random, you are also bound to find significant predictor variables simply by chance. You should consider adjusting for multiple comparisons. Really you should have some hypotheses you are testing rather than just searching for something that gives you a low p-value.
Also, since it sounds like you don't care what the final model looks like, you may want to try glmulti() from the glmulti package. It will create every possible model given your data and eventually spit out the "best" one.
A: Use mutual information. Estimating it between categorical and continuous variables is not straightforward, but there is plenty of literature on the matter.
In R, you can use the entropy package.
For feature selection there exist algorithms that are more sophisticated than simple correlation, which can be used for categorical if you "explode" the features (e.g. turn a 3-class variable into 3 boolean ones). Take a look at filter and wrapper methods. From an algebraic perspective you have approximate solutions to the Column Subset Selection Problem and Rank-Revealing QR factorizations.
