AUC for random classifier in case of unbalanced dataset

Question

If my dataset is highly unbalanced say 90% negative data point and 10% positive data point , would using a random classifier give a AUC value of 0.5 ?

Answer 1

A random classifier gives AUC 0.5 in expectation regardless of class balance.

@article{Fawcett:2006:IRA:1159473.1159475,
 author = {Fawcett, Tom},
 title = {An Introduction to ROC Analysis},
 journal = {Pattern Recogn. Lett.},
 issue_date = {June 2006},
 volume = {27},
 number = {8},
 month = jun,
 year = {2006},
 issn = {0167-8655},
 pages = {861--874},
 numpages = {14},
 url = {http://dx.doi.org/10.1016/j.patrec.2005.10.010},
 doi = {10.1016/j.patrec.2005.10.010},
 acmid = {1159475},
 publisher = {Elsevier Science Inc.},
 address = {New York, NY, USA},
 keywords = {Classifier evaluation, Evaluation metrics, ROC analysis},
} 

Answer 2

Yes, but see bellow. One of the advantages of AUC is precisely that it measures the classification accuracy regardless of how many positives and negatives there are.

AUC is the Area Under the ROC curve. The ROC curve plots the False Positive Rate against the True Positive Rate, with the False Positive Rate being the ratio of misclassified negative cases (i.e. were negative and were labeled as positive divided by the number of total negative cases) and the True Positive Rate the ratio of correctly classified positive cases (i.e. were positive and were correctly classified as positive, divided by the number of positive cases).

So, to your question:

If my dataset is highly unbalanced say 90% negative data point and 10% positive data point , would using a random classifier give a AUC value of 0.5 ?

Yes, I have marked in boldface the answer.

---

Also, beware, you have to understand random as a classifier that randomly classifies the data but using the probability distribution of the data. For example, say you have a dataset with:

$90$ positives $(1)$, $10$ negatives $(0)$.

Random classifier here means a generator of cases with probabilities $P(1) = 0.9$ and $P(0)=0.1$. Note that if you simply generate $1$s and $0$s without taking into account the probability you will have a lower AUC.

Also note that this is, as Sycorax has stated, in expectation, so when the number of cases goes to $\infty$.

Answer 3

When there is a class imbalance, we must seek the business domain experts (if we are not) and see what they are after. are they after negative or positive data points. once that is freezed, and a go ahead for fixing a class imbalance, there are 7 major techniques to hanle.. please refer https://www.kdnuggets.com/2017/06/7-techniques-handle-imbalanced-data.html

please note, tuning your model to perform for a particular accuracy is as good as killing the data.. if i did not mi understood your question..

class imbalance may or may not increase your accuracy.. so far i haven't seen any improvement, but if any i accept it as my learning..

leaving this to experts in this community :)