# a general measure of data-set imbalance

I am working on thousands of datasets, many of them are "unbalanced" ; either a multi-class list with highly skewed distribution (For example three categories with ratio of 3500:300:4 samples) or a continues number with skewed distribution. I am looking for some metric that can say "How badly unbalanced" the dataset is. Is there such a meteric?

Eventually I want to score these datasets according to their balanced metric and provide a different balancing/ machine learning solution to each of them. I would prefer a python solution if exist.

• If you use R, you can use the diversity() function in the 'vegan' package to calculate the Shannon-Weaver index. rdocumentation.org/packages/vegan/versions/2.4-2/topics/… – Zilu Liang Feb 1 at 4:42
• It's not clear why you chose the word balance as if imbalance is a bad thing. If you use a method that depends on the amount of balance in levels of $Y$, switch to another method. – Frank Harrell Feb 7 at 11:59

You could use the Shannon entropy as a measure of balance.

On a data set of $n$ instances, if you have $k$ classes of size $c_i$ you can compute entropy as follows: $$H = -\sum_{ i = 1}^k \frac{c_i}{n} \log{ \frac{c_i}{n}}.$$

This is equal to:

• $0$ when there is one single class. In other words, it tends to $0$ when your data set is very unbalanced
• $\log{k}$ when all your classes are balanced of the same size $\frac{n}{k}$

Therefore, you could use the following measure of Balance for a data set: $$\mbox{Balance} = \frac{H}{\log{k}} = \frac{-\sum_{ i = 1}^k \frac{c_i}{n} \log{ \frac{c_i}{n}}. } {\log{k}}$$ which is equal to:

• $0$ for a unbalanced data set
• $1$ for a balanced data set
• Note that any logarithm base is fine because the normalization makes the formula invariant to the base. – Simone Oct 13 '16 at 11:36
• @Simone, do you know if this balance equation is a standard balance measure in the literature? – felipeduque Feb 16 '17 at 20:44
• Not in particular. I think I saw using entropy as measure of balance in literature. Other times I saw using $\min{\{c_i\}}/\max{\{c_i\}}$. I guess it does not really matter. They are usually just used as guidance to check the degree of balance of a data set. – Simone Feb 17 '17 at 9:35
• @Simone, interesting. Other two options can be using KL distance or cross entrpoy to measure the "distance" of {c_i}/n probabilities to 1/k. For KL distance this should be zero of the data-set is balanced and for cross entropy it should be come the log(k) for balanced dataset – oak Oct 29 '18 at 14:19
• Simone's suggestion is referred to as the "Shannon Diversity Index" and is common in ecology research. Here is some more information: itl.nist.gov/div898/software/dataplot/refman2/auxillar/… – James Budarz Nov 5 '20 at 23:21

Based on the answer of Simone, I wrote this short python code to calculate balance, which works very well for me.

def balance(seq):
from collections import Counter
from numpy import log

n = len(seq)
classes = [(clas,float(count)) for clas,count in Counter(seq).items()]
k = len(classes)

H = -sum([ (count/n) * log((count/n)) for clas,count in classes]) #shannon entropy
return H/log(k)


Thank you very much!

• for me, I had to change .iteritems() for .items() only, then it worked! thanks! – Caio Oliveira Aug 13 '20 at 21:31