5
$\begingroup$

If my dataset is highly unbalanced say 90% negative data point and 10% positive data point , would using a random classifier give a AUC value of 0.5 ?

$\endgroup$
3
$\begingroup$

A random classifier gives AUC 0.5 in expectation regardless of class balance.

@article{Fawcett:2006:IRA:1159473.1159475,
 author = {Fawcett, Tom},
 title = {An Introduction to ROC Analysis},
 journal = {Pattern Recogn. Lett.},
 issue_date = {June 2006},
 volume = {27},
 number = {8},
 month = jun,
 year = {2006},
 issn = {0167-8655},
 pages = {861--874},
 numpages = {14},
 url = {http://dx.doi.org/10.1016/j.patrec.2005.10.010},
 doi = {10.1016/j.patrec.2005.10.010},
 acmid = {1159475},
 publisher = {Elsevier Science Inc.},
 address = {New York, NY, USA},
 keywords = {Classifier evaluation, Evaluation metrics, ROC analysis},
} 
$\endgroup$
  • $\begingroup$ I think the question is more basic than that. Of course, it is expectation, but it is not what the user is asking, in my opinion. $\endgroup$ – D1X Apr 17 '18 at 16:21
  • $\begingroup$ It's hard to see how the question could be otherwise than the single sentence in the text - and if OP has other basic questions, the cited article is a good place to start looking for answers. $\endgroup$ – Sycorax says Reinstate Monica Apr 17 '18 at 16:29
1
$\begingroup$

Yes, but see bellow. One of the advantages of AUC is precisely that it measures the classification accuracy regardless of how many positives and negatives there are.

AUC is the Area Under the ROC curve. The ROC curve plots the False Positive Rate against the True Positive Rate, with the False Positive Rate being the ratio of misclassified negative cases (i.e. were negative and were labeled as positive divided by the number of total negative cases) and the True Positive Rate the ratio of correctly classified positive cases (i.e. were positive and were correctly classified as positive, divided by the number of positive cases).

So, to your question:

If my dataset is highly unbalanced say 90% negative data point and 10% positive data point , would using a random classifier give a AUC value of 0.5 ?

Yes, I have marked in boldface the answer.


Also, beware, you have to understand random as a classifier that randomly classifies the data but using the probability distribution of the data. For example, say you have a dataset with:

$90$ positives $(1)$, $10$ negatives $(0)$.

Random classifier here means a generator of cases with probabilities $P(1) = 0.9$ and $P(0)=0.1$. Note that if you simply generate $1$s and $0$s without taking into account the probability you will have a lower AUC.

Also note that this is, as Sycorax has stated, in expectation, so when the number of cases goes to $\infty$.

$\endgroup$
-1
$\begingroup$

When there is a class imbalance, we must seek the business domain experts (if we are not) and see what they are after. are they after negative or positive data points. once that is freezed, and a go ahead for fixing a class imbalance, there are 7 major techniques to hanle.. please refer https://www.kdnuggets.com/2017/06/7-techniques-handle-imbalanced-data.html

please note, tuning your model to perform for a particular accuracy is as good as killing the data.. if i did not mi understood your question..

class imbalance may or may not increase your accuracy.. so far i haven't seen any improvement, but if any i accept it as my learning..

leaving this to experts in this community :)

$\endgroup$
  • 5
    $\begingroup$ The question is what would the value of the AUC be if you classify at random from a two-class problem with unbalance in the data. You don't answer that question. $\endgroup$ – Michael R. Chernick Apr 17 '18 at 15:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.