I have two different sources of data and I would like to check if their class distributions are similar and how much.

The first dataset ($D_1$) has 2M samples and the second ($D_2$) 6M.

When I calculate counts per class, they have exponential distribution - many rare classes and a small number of very frequent ones.

I would like to do the comparison twice based on 2 variables. In the first case there are 100 classes and in the second 5500.

I am thinking about using chi-square (https://docs.scipy.org/doc/scipy-0.15.1/reference/generated/scipy.stats.chi2_contingency.html) or G-test. However, they should be used with a normal distribution. Thus, I wonder if I can use log1p transformation or not.

Another idea which I have is calculating KL distance between the class frequencies and the distance to uniform distribution to have a reference.

Despite the mentioned tests, I am interested in what is a correct approach to compare the class distribution in the mentioned situation.

EDIT (in reply to whuber comment):

Yes, I mean here the discrete values. The raw data are like labels e.g. [A, A, B, A, C, C]. Then I transformed it to counts: {A: 3, B: 1, C: 2}. By exponential, I meant that when I check the number of samples for each class, the histogram looks like: Class size distribution

  • 1
    $\begingroup$ Could you clarify for us what a "class distribution" is and in what sense counts (which by their very nature take on discrete values) can be considered to have "exponential distribution," which usually refers to a specific family of continuous distributions? $\endgroup$ – whuber Jan 16 at 14:06
  • $\begingroup$ Your plot looks more skew than an exponential distribution. $\endgroup$ – Glen_b -Reinstate Monica Jan 17 at 5:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.