7
$\begingroup$

Thanks in advance for the help.

I have an unbalanced dataset that I am using for a binary classification problem. The classes are unbalanced. I believe that in such a case that weighted area under roc (receiver operating curve) is the proper way to analyze my results. However, I don't full understand the difference between area under roc and weighted area under roc.

$\endgroup$
10
$\begingroup$

One of the advantages to ROC curves is that they are agnostic to class skew. ROC curves remain the same whether your data is balanced or not, bar some finite-sample effects when you have very few examples of one class.

As such, weighted ROC curves have nothing to do with class balance. Instead, weighted ROC curves are used when you're interested in performance in a certain region of ROC space (e.g. high recall) and was proposed as an improvement over partial AUC (which does exactly this but has some issues). You can read more about it in Weighted Area Under the Receiver Operating Characteristic Curve and Its Application to Gene Selection by Li and Fine.

| cite | improve this answer | |
$\endgroup$
  • 1
    $\begingroup$ I don't know if the OP meant the weighted AUC used in WEKA. In that case it is just an extension of the AUC, valid for binary class problems, to multiclass problems. $\endgroup$ – Simone Jun 27 '15 at 22:20
  • $\begingroup$ weighted AUC (wAUC), is a better way to measure the imbalanced data learning classifiers. When computing the weighted area under the ROC curve, weights vary with the values of the true positive rate (TPrate) among regions in a bid to focus on the accuracy of minority class that is more important in common. As AUC is a special case of wAUC, wAUC is compared with other common performance evaluation measures by isometric analysis. The experimental results show that wAUC can distinguish the classifiers with the same AUC values. $\endgroup$ – jperezmartin May 19 at 22:36
1
$\begingroup$

I second Manish Mahajan's suggest for looking into SMOTE, which can improve your ROC by synthesizing data from the minority class. It is a researched technique and not as simple as "a fictitious set of data" as user48956 put it. See the paper.

| cite | improve this answer | |
$\endgroup$
0
$\begingroup$

Hello if you have class imbalance then the approach usually involves balancing the class either by oversampling or under sampling or create synthetic samples using the SMOTE algorithm. Once this is done then the usual Area under curve of ROC can be looked at. Hope that makes sense I am afraid I haven't yet used weighted ROC so not sure about that approach

| cite | improve this answer | |
$\endgroup$
  • 2
    $\begingroup$ Under/oversampling is great for improving the learner, but using it to generate training data means that you're evaluating your algorithms on a fictitious set of data -- the score has no interpretation in application. $\endgroup$ – user48956 Apr 26 '17 at 17:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.