# Compare two discrete distributions using Chi-Sq

(I am not quite sure whether the following post is better suited for CrossValidated or for StackOverflow because I have both a theoretical question about the use of the Chi-Squared test and a more practical question about its implementation in R.)

I would like to test whether a given discrete distribution of frequency counts is significantly different from an expected background distribution.

The observed distribution is based on the frequency counts of a set of 420 given variables. I generated the expected background distribution by summing up the frequency counts for 1000 sets of 420 randomly chosen variables.

Now I need to know whether there is a difference between the two distributions. Is it correct to use the Chi-Squared test for this problem? The values for which the frequency counts are given go from 0 to 50 but several of the counts are small/equal to zero.

I already tried to use the chisq.test{stats} R function but didn’t get any result:

> expected <- c(49202,79371,77037,63164,48844,34097,23999,16521,11022,6200,4857,2866,1382,500,225,0,135,41,35,0,94,0,0,36,0,33,0,0,49,44,0,48,39,41,0,33,0,0,0,0,0,0,0,43,0,0,0,0,0,0,42)
> observed <- c(66,85,64,56,37,25,26,24,13,5,7,4,2,2,0,2,0,1,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0)
> freq_table <- cbind(expected, observed)
> rownames(freq_table) <- seq(0,50)
> chisq.test(freq_table)

Pearson's Chi-squared test

data:  freq_table
X-squared = NaN, df = 50, p-value = NA


Just remove rows that contain only zeros:

> expected <- c(49202,79371,77037,63164,48844,34097,23999,16521,11022,6200,4857,2866,1382,500,225,0,135,41,35,0,94,0,0,36,0,33,0,0,49,44,0,48,39,41,0,33,0,0,0,0,0,0,0,43,0,0,0,0,0,0,42)
> observed <- c(66,85,64,56,37,25,26,24,13,5,7,4,2,2,0,2,0,1,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0)
> freq_table <- cbind(expected, observed)
> freq_table_rowSums<-rowSums(freq_table)
> freq_table2<-freq_table[freq_table_rowSums>0,]
> chisq.test(freq_table2)

Pearson's Chi-squared test

data:  freq_table2
X-squared = 2065,6, df = 29, p-value < 0,00000000000000022


Why rows filled with zeros only are forbidden? Because such row has marginal sum equal to 0, which in case, causes all expected values in this row to be zeros. And this leads to division by zero in chi-squared statistic.

• Can you explain this? – Sextus Empiricus Nov 6 '18 at 12:52
• I added some explanation to an answer – Łukasz Deryło Nov 6 '18 at 12:57
• That seems like a good hack to me, but it is also a simplistic solution. What if one of those zero's is a relevant difference/anomaly? – Sextus Empiricus Nov 6 '18 at 13:19
• Row of zeros means that there is a level of row variable that does not appear in your data. So maybe this anomaly means that you should change your sampling scheme or variable definition? – Łukasz Deryło Nov 6 '18 at 14:42