# How to test for differences between two distributions with percentages?

Imagine there are two distributions:

Distribution 1 lists the shares of each ice cream flavor of all favourite flavors in village A:

Vanilla 20%
Chocolate 60%
Strawberry 20%


Distribution 2 lists the shares of each ice cream flavor of all favourite flavors in village B:

Vanilla 30%
Chocolate 40%
Strawberry 30%


How can I test these two lists for significant differences? I think a Chi-squared-test is not possible, because the values are percentages.

• If all you have are percentages then there's no way to do a formal test since you have no information on sample size. – dsaxton Aug 16 '18 at 13:45
• The round numbers suggest a very small sample size. Do you know the sample size? If so, you can run a chi sq. – Joel W. Aug 16 '18 at 14:07
• @JoelW. It's a notional distribution. My original sample size is much larger. It contains more than 600 samples. – BeneGIS Aug 16 '18 at 14:11
• With an N of 600, you will be able to conduct a chi-square test, absent very unusual circumstances (like many flavors and low frequencies). – Joel W. Aug 16 '18 at 18:11

If you have 600 respondents in each city, then you can do a chi-squared test for homogeneity (of distributions in the two cities). In R, the basic computation looks like this:

DTA = matrix(c(120,180, 360,240,  120,60,  0,120), byrow=T, nrow=4)
DTA
[,1] [,2]
[1,]  120  180
[2,]  360  240
[3,]  120   60
[4,]    0  120
chisq.test(DTA)

Pearson's Chi-squared test

data:  DTA
X-squared = 176, df = 3, p-value < 2.2e-16


The small P-value indicates people in the two cities have different preferences for flavors of ice cream. I will leave it to you to find the expected counts in each cell and to compute the chi-squared statistic. For the so-called chi-squared statistic to have approximately the distribution $\mathsf{Chisq}(df = 2),$ all of the expected counts should be above 5. (Some authors say all above 3 and most above 5.)

However, if you have only 20 subjects in each city, then the data table and chi-squared test are as shown below:

DTA2 = DTA/30
DTA2
[,1] [,2]
[1,]    4    6
[2,]   12    8
[3,]    4    2
[4,]    0    4

chisq.test(DTA2, sim=T)

Pearson's Chi-squared test with simulated p-value (based on 2000 replicates)

data:  DTA2
X-squared = 5.8667, df = NA, p-value = 0.1229


Here not all of the expected counts are sufficiently large. For example, $$E_{41} = E_{42} = \frac{4(20)}{40} = 2 < 5.$$ Thus, the chi-squared statistic does not have distribution $\mathsf{Chisq}(df = 2).$ In R statistical software, the procedure chisq.test can simulate the distribution of the chi-squared statistic and provide an approximate P-value; here above 5%.

There is much less information with only 20 subjects in each city. Unfortunately, for the (presumably fake) percentages you provided, differences are not large enough that the test can find a significant difference in the preference distribution between the two cities.

Notes:

(a) Clusters of bar charts for the two cities would look the same (except for numbers on axes) in our examples whether we have 1200 subjects overall or 40. In general, it is not possible to tell from looking at bar charts whether the two cities differ. Such bar charts should always have footnotes with results of chi-squared tests.

(b) For a chi-squared test, the degrees of freedom depend on the number of levels of each categorical variable [here $(r-1)(c-1) = 3)],$ not on the sample sizes.