4
$\begingroup$

Thanks in advance for the help.

I have an unbalanced dataset that I am using for a binary classification problem. The classes are unbalanced. I believe that in such a case that weighted area under roc (receiver operating curve) is the proper way to analyze my results. However, I don't full understand the difference between area under roc and weighted area under roc.

$\endgroup$
7
$\begingroup$

One of the advantages to ROC curves is that they are agnostic to class skew. ROC curves remain the same whether your data is balanced or not, bar some finite-sample effects when you have very few examples of one class.

As such, weighted ROC curves have nothing to do with class balance. Instead, weighted ROC curves are used when you're interested in performance in a certain region of ROC space (e.g. high recall) and was proposed as an improvement over partial AUC (which does exactly this but has some issues). You can read more about it in Weighted Area Under the Receiver Operating Characteristic Curve and Its Application to Gene Selection by Li and Fine.

$\endgroup$
  • 1
    $\begingroup$ I don't know if the OP meant the weighted AUC used in WEKA. In that case it is just an extension of the AUC, valid for binary class problems, to multiclass problems. $\endgroup$ – Simone Jun 27 '15 at 22:20
1
$\begingroup$

I second Manish Mahajan's suggest for looking into SMOTE, which can improve your ROC by synthesizing data from the minority class. It is a researched technique and not as simple as "a fictitious set of data" as user48956 put it. See the paper.

$\endgroup$
0
$\begingroup$

Hello if you have class imbalance then the approach usually involves balancing the class either by oversampling or under sampling or create synthetic samples using the SMOTE algorithm. Once this is done then the usual Area under curve of ROC can be looked at. Hope that makes sense I am afraid I haven't yet used weighted ROC so not sure about that approach

$\endgroup$
  • 2
    $\begingroup$ Under/oversampling is great for improving the learner, but using it to generate training data means that you're evaluating your algorithms on a fictitious set of data -- the score has no interpretation in application. $\endgroup$ – user48956 Apr 26 '17 at 17:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.