4
$\begingroup$

I am comparing classification algorithms on different data-sets. For some of the algorithms, there are reasons to believe they don't work well with class imbalance unless some kind of adjustment is made. If the adjustments work well, the experiments on the imbalanced sets should show that the adjustment helps and the balanced sets will tell me if the adjustment remains at least neutral (not harmful) where it is not really needed.

I would like to introduce the used data-sets in a table with columns for the number of records, variables and classes and also one column that measures the class imbalance.

Potential measures that came to my mind where the ratio between the prevalence of the most and the least common class. I also thought about using something like the Herfindahl index. It does not correct for the number of classes, which in economics it shouldn't, but I would need it to do.

Do you have any suggestions? I would like to avoid listing the prevalences of all classes for all data-sets since the number of classes differs.

$\endgroup$
1
$\begingroup$

Potential measures that came to my mind where the ratio between the prevalence of the most and the least common class.

I think this is the right idea. Actually, I would suggest that you use two numbers instead of just one, and simply present the prevalence of the most and least common classes. This is direct and easy to understand.

$\endgroup$
  • $\begingroup$ I will do this unless something else comes up. I'm just a little hesitant since the reader would need to compare this in his head to the expected prevalence under perfect balance. $\endgroup$ – David Ernst Nov 3 '16 at 18:34
  • $\begingroup$ @user7019377 I don't think they would need to, actually. The important thing is just how the min and max compare, not how either of these compares to the reciprocal of the number of classes. $\endgroup$ – Kodiologist Nov 4 '16 at 5:32
  • $\begingroup$ @DavidErnst Did you did this finnaly? $\endgroup$ – ucsky Nov 14 '17 at 10:25
  • $\begingroup$ Yes it worked well $\endgroup$ – David Ernst Nov 14 '17 at 13:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.