12
$\begingroup$

i have a data set that is strictly binary. each variable's set of values is in the domain: true, false.

the "special" property of this data set is that an overwhelming majority of the values are "false".

i have already used a bayesian network learning algorithm to learn a network from the data. however, for one of my target nodes (the most important one, being death), the AUC result is not very good; it is slightly better than chance. even the positive-predictive value (PPV), which has been suggested to me on CV, was not competitive with what's reported in the literature with other approaches. note that AUC (ROC analysis) is the typical benchmark reported in this area of clinical research, but i am also opened to suggestions on how to more appropriately benchmark the classification model if there are any other ideas.

so, i was wondering what other classification models i can try for this type of data set with this property (mostly false values).

  • would support vector machine help? as far as i know, SVM only deals with continuous - variables as the predictors (although it has been adapted to multi-class). but my variables are all binary.
  • would a random forest help?
  • would logistic regression apply here? as far as i know, the predictors in logistic regression also are continuous. is there a generalized version for binary variables as the predictors?

aside from classification performance, i suspect SVM and random forest might very well outperform the bayesian network, but the problem shifts to how to explain the relationships in these models (especially to clinicians).

Improve this question
$\endgroup$
6
  • $\begingroup$ This has been asked many times, I have answered similar questions here: stats.stackexchange.com/questions/78469/… and here: stats.stackexchange.com/questions/67755/… and as to how interpret the outputs, you should check out how to interpret marginal effects of your explanatory variables on your target variable. Check out for example: hosho.ees.hokudai.ac.jp/~kubo/Rdoc/library/randomForest/html/… $\endgroup$
    – JEquihua
    Mar 14, 2014 at 15:35
  • $\begingroup$ If your data is very sparse and generally pretty poor, you might want to look an nearest neighbors classifier. Though make sure to weigh your features properly. $\endgroup$
    – Akavall
    Mar 15, 2014 at 1:32
  • $\begingroup$ @Akavall could you please give some pointers on weighting the features properly? they are all binary (predictors and class variable). i would like to stick with PPV as the primary weight, but i could also use mutual information as well. or i suppose i can use any number of contingency table association analysis. $\endgroup$
    – Jane Wayne
    Mar 16, 2014 at 22:42
  • $\begingroup$ @JaneWayne, nearest neighbors does nothing in terms of features selection/feature weighting; if the features are bad or weighted improperly the algorithm would do really bad, on the other hand if the features weighted properly this simple algorithm can do really well. However, weighting properly is not easy. And you current solution might be good already. If you know something about the data set you could manually assign bigger weights to some features. Or if you are able to evaluate performance of the model at different times, you can adapt some kind of learning heuristic algorithm to choose $\endgroup$
    – Akavall
    Mar 18, 2014 at 14:29
  • $\begingroup$ features based of performance. However, here you have to assume that the objective function you are trying to maximize is relatively smooth, and there is a cost due to exploration and exploitation trade off. $\endgroup$
    – Akavall
    Mar 18, 2014 at 14:30

2 Answers 2

Reset to default
5
$\begingroup$

would support vector machine help? as far as i know, SVM only deals with continuous - variables as the predictors ...

Binary variables are not a problem for SVM. Even specialized kernels exist for exactly such data (Hamming kernel, Tanimoto/Jaccard kernel), though I don't recommend using those if you're not intimately familiar with kernel methods.

would logistic regression apply here? as far as i know, the predictors in logistic regression also are continuous

Logistic regression works with binary predictors. It is probably your best option.

how to explain the relationships in these models (especially to clinicians).

If you use linear SVM it is fairly straightforward to explain what's going on. Logistic regression is a better option, though, since most clinicials actually know these models (and by know I mean have heard of).

Improve this answer
$\endgroup$
2
$\begingroup$

I would like to share my experiment of classifying about .3 million binary data with a majority of false values. I have used Linear SVM, Complex trees, LDA, QDA, logistic regression etc. All these methods had an efficiency of about 54%, which is not good. According to my professor, the classification methods that could help me in this problem are Neural Networks, Quadratic SVM but I haven't tested these. I hope this could help.

Improve this answer
$\endgroup$

Not the answer you're looking for? Browse other questions tagged or ask your own question.