1
$\begingroup$

I need to utilize two different classifier to get best classification results. Since, it seems that they complement each other (not sure I am not expert btw). ROC characteristics are given below (testing scheme is 10-fold cross validation):

MetaCost [0 8; 1 0] Alternating Decision Tree (ADTree)

               TP Rate   FP Rate   Precision   Recall  F-Measure   ROC Area  Class
                 0.973     0.62       0.119     0.973     0.212      0.696    YES
                 0.38      0.027      0.994     0.38      0.55       0.696    NO
Weighted Avg.    0.427     0.074      0.925     0.427     0.523      0.696

MetaCost [0 8; 1 0] Logistic

               TP Rate   FP Rate   Precision   Recall  F-Measure   ROC Area  Class
                 0.604     0.161      0.245     0.604     0.348      0.771    YES
                 0.839     0.396      0.961     0.839     0.896      0.769    NO
Weighted Avg.    0.821     0.377      0.904     0.821     0.853      0.769

I have tried Voting but could not get desired results. Therefore, it is time to seek for expert help. Could you please advise me a solution, if possible?

Thanks in advance. Also, as a reminder I am not an expert.

EDIT: Best I can get:

Vote combines the probability distributions of these base learners:
    weka.classifiers.meta.AdaBoostM1 -P 100 -S 1 -I 10 -W weka.classifiers.meta.MetaCost -- -cost-matrix "[0.0 8.0; 1.0 0.0]" -I 10 -P 100 -S 1 -W weka.classifiers.trees.ADTree -- -B 10 -E -3
    weka.classifiers.meta.AdaBoostM1 -P 100 -S 1 -I 10 -W weka.classifiers.meta.MetaCost -- -cost-matrix "[0.0 8.0; 1.0 0.0]" -I 10 -P 100 -S 1 -W weka.classifiers.functions.Logistic -- -R 1.0E-8 -M -1
using the 'Product of Probabilities' combination rule 

               TP Rate   FP Rate   Precision   Recall  F-Measure   ROC Area  Class
                 0.706     0.204      0.231     0.706     0.348      0.825    YES
                 0.796     0.294      0.969     0.796     0.874      0.825    NO
Weighted Avg.    0.789     0.287      0.91      0.789     0.832      0.825
$\endgroup$
  • 1
    $\begingroup$ Why these two? Ensembles of several classifiers even of the same type are often better than any single one. $\endgroup$ – Michael R. Chernick May 21 '12 at 14:50
  • 1
    $\begingroup$ @Michael Chernick, Since, Logistic seems better classifier for YES, whereas ADTree seems better for NO. I have tried Boosting logistic, it did not worked either. $\endgroup$ – baris.aydinoz May 21 '12 at 14:53
  • $\begingroup$ Have you ever used Random Forests? If ensemble averages don't work why would combining these two be promising? $\endgroup$ – Michael R. Chernick May 21 '12 at 17:03
  • $\begingroup$ @Michael Chernick, yes, but there must be a way to combine these to in a correct manner. $\endgroup$ – baris.aydinoz May 21 '12 at 17:28
  • 1
    $\begingroup$ Possible duplicate of How to combine weak classfiers to get a strong one? $\endgroup$ – kjetil b halvorsen Aug 14 '18 at 22:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.