Classification problem-Big Data and simple decision rules: logit regression, LDA, random forest, cond. trees, or something else?

Question

This is a big data question from someone who is more accustomed to small data. I would like to develop some classification "rules of thumb," that is, some simple decision rules or a decision tree that would allow end-users to improve classification on the binary outcome. Such rules might include, for example, something like "If X1 * X2 > 10.5, predict outcome=failure" or "Predict outcome=success if both X1 < 10.1 and X3 < 5.2, or if X5 < 7.3." I have n=1,000,000 observations, p=32 continuous predictors, and one binary outcome that consists of approximately 90% successes, 10% failures overall.

I have tried various logistic regression models. Not surprisingly, the large sample causes most predictor variables to be significant, but when I use the logistic regression model results for prediction in a hold-out validation sample, I get only barely better prediction accuracy than the mindless rule of always predicting success. Perhaps worse, these models' prediction rules are so complex that they would never be adopted by end-users.

What statistical tool(s) would be most appropriate for this problem?

Answer 1

I would suggest you try support vector machines. A polynomial kernel could capture the interacting features in your example without you having to explicitly enumerate them. LibSVM is probably the most widely used implementation.

There may be better choices depending on the distribution of your data points. There is a helpful illustration of different classifiers on three different datasets this page.