When is dataset considered unbalanced?

Question

I have data set which is highly unbalanced - target attribute is 93% False and 7% True. But I know that this is normal for my kind of data.

I am afraid that if I undertake any steps (I can take less False cases for example), I skew the distribution - the model will see True class as more frequent and give it higher probability, which in reality is not true.

The question

Can I say that my dataset is unbalanced even if it represents reality? Or we talk about unbalanced data only if we took bad sample and in reality the data are balanced?

(I am asking because I am estimating the target attribute and I am considering if I should do something with my dataset regarding the unbalanced nature of it)

Answer 1

While the distribution you mention is normal for your field, this is still an unbalanced data set. I don't think there is anything bad being an unbalanced data set. We label something about unbalanced just because the distribution is highly asymmetry not because the data is a "bad sample". Indeed, unbalanced data set is very common.

All you need to know is that your sample is unbalanced, thus you should exercise caution for your statistics. For example, precision would not be an appropriate statistic for you because I could easily make a predictor that always predicts False. A better metric would have been ROC or F1-score.

Furthermore, some learning algorithm such as decision tree is biased for unbiased data. You just need to understand what you are doing.