2
$\begingroup$

This question might be trivial, but I have problems understanding this line taken from here:

The balanced_accuracy_score function computes the balanced accuracy, which avoids inflated performance estimates on imbalanced datasets. It is the macro-average of recall scores per class or, equivalently, raw accuracy where each sample is weighted according to the inverse prevalence of its true class. Thus for balanced datasets, the score is equal to accuracy.

I have particular problems with the last line. So let's take the definition as sklearn provides it: $$\text{bal}\_{\text{acc}} := \frac{1}{2}\left(\frac{\text{TP}}{\text{TP}+\text{FN}} + \frac{\text{TN}}{\text{TN+FP}}\right)$$ The way I understood it, for a balanced dataset, $\text{TP} = \text{TN}$. Therefore, the last Eq. can be rewritten to: $$\text{bal}\_{\text{acc}} := \frac{1}{2}\left(\frac{\text{TP}}{\text{TP}+\text{FN}} + \frac{\text{TP}}{\text{TP+FP}}\right) = \frac{1}{2}\left(\frac{\text{TP}\left(\text{TP}+\text{FP}\right)}{\left(\text{TP}+\text{FN}\right)\left(\text{TP}+\text{FP}\right)} + \frac{\text{TP}\left( \text{TP}+\text{FN}\right)}{\left(\text{TP+FP}\right)\left(\text{TP}+\text{FN}\right)}\right) = \frac{1}{2}\frac{\text{TP}\left( \text{TP}+\text{FN}\right) + \text{TP}\left(\text{TP}+\text{FP}\right)}{\left(\text{TP+FP}\right)\left(\text{TP}+\text{FN}\right)}$$


Unfortunately, I do not see how we can recover the "classical" accuracy: $$\text{acc} := \frac{\text{TP}+\text{TN}}{\text{TP}+\text{TN}+\text{FP}+\text{FN}}$$ Assuming that $\text{TP}=\text{TN}$ again, we have: $$\text{acc} = \frac{2\text{TP}}{2\text{TP}+\text{FP}+\text{FN}}$$ Could anybody help, please?

$\endgroup$
2

1 Answer 1

1
$\begingroup$

Since no real answer came to this I will just expand on my comment above:

"Balance" is a property of the dataset: there are as many positives in total as there are negatives in total. That is TP + FN = TN + FP. TP (true positives) and TN (true negatives) are properties of your classifier: they are the number of positives and negatives that it gets right.

If we take TP + FN = TN + FP we have

$$ \frac{1}{2} \left(\frac{TP}{TP + FN} + \frac{TN}{TN + FP}\right) = \\ \frac{1}{2}\left(\frac{TP}{TP + FN} + \frac{TN}{TP + FN}\right) = \\ \frac{1}{2}\left(\frac{TP + TN}{TP + FN}\right) = \\ \frac{1}{2}\left(\frac{TP + TN}{\frac{1}{2}(TP + FN + TN + FP)}\right) = \\ \frac{TP + TN}{TP + FN + TN + FP}, $$

as per the quoted text.

My view, for what it's worth, is that "imbalanced data" is a fake problem produced by unwarranted focus on accuracy as a target for model selection. I encourage you to check out @Stephan Kolassa's links in the comments.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.