# Chi squared test with groups of different sample size

I have two groups of unequal size, one with 10 and the other with 5 persons. Both groups are given a coin and each person flips this coin an unknown, large, but fixed number of times and reports the number of heads.

The question is how can one determine if the groups had the same coin or if one of them got a coin that was biased? If the number of persons where the same in both groups one could just compare them with a chi-squared test but how can this be done with unequal group sizes?

Here is the R code for creating the experiment:

set.seed(1234)

unknown <- 100

coin1 <- rbinom(10, unknown, 1/4)
coin2 <- rbinom(5, unknown, 1/2)

• Even with equal sample sizes, I wouldn't use a chi-squared test for this experiment. (equal sample sizes is not a requirement for a chi-square test). I agree with Phil, this is a question better suited to Cross Validated. – Benjamin Dec 17 '15 at 12:57

The null hypothesis is that neither group has a biased coin (and so p=0.5 for both); the alternative is that one of them has a biased coin (so p differs from 0.5 for one of the groups).

Let the unknown number of tosses be $t$. Let the number of people in group 1 be $n_1$, the number in group 2 be $n_2$.

Under the null hypothesis, the total number of heads in each of the groups is $\text{binomial}(tn_i,0.5)$, while the total of both groups together is $\text{binomial}(t(n_1+n_2),0.5)$

If you condition on the total number of heads observed, $n_h=n_1+n_2$, then under the null hypothesis the number of those observed heads that came from group 1 is also $\text{binomial}(tn_h,\frac{n_1}{n_h})$

As a result, one possible approach is that you could either do a one-sample proportions test or a chi-square goodness of fit test (they're effectively the same test).

I'm still not sure I would approach this with a chi-square test. Glen_b uses an interesting approach that recognizes that a binomial experiment is nothing more than a series of Bernoulli experiments. Due to the independence assumption, you can string out all of those Bernoulli experiments into one giant experiment and use the chi-square test to compare the proportion of heads observed to the proportion of heads expected.

Mathematically, it's sound, but I'm not crazy about it because it doesn't directly address the question of whether both groups are using similarly weighted coins. And while your example uses a group with a fair coin, your description of the problem, "similarly weighted coin" suggests that we may not always know the weight of the coins. But when we know the weight of at least one coin, my complaint of not directly addressing the question of whether both groups are using similarly weight coins is probably a po-tay-to/po-tah-to argument. You could always assume that the observed proportion from one group is the population proportion, but you'd probably be better off doing a two sample test of proportions at that point.

Personally, I would favor Poisson regression in this situation. (Disclaimer: negative binomial regression is probably a better choice, but when I tried to build models with it, I was getting rather poor fits. That could either be user error on my part, or that the data I'm using just aren't fitting well in a negative binomial model--which usually means Poisson is a suitable fit). A Poisson regression will give similar conclusions, but doesn't require making assumptions about the weighting of the coins. It will test if there is evidence to conclude that the heads counts are different based on the number of observed heads in each group.

A comparison of results from your sample code using a one sample proportion test and poisson regression:

set.seed(1234)

compareTests <- function(n1, n2, p1, p2, tosses, binom_null = p1)
{
coin1 <- rbinom(n1, tosses, p1)
coin2 <- rbinom(n2, tosses, p2)

p.chisq <- prop.test(x = sum(coin1) + sum(coin2),
n = (length(coin1) + length(coin2)) * tosses,
p = binom_null)\$p.value

DFrame <- data.frame(heads = c(coin1, coin2),
coin = rep(c("Coin 1", "Coin 2"),
c(length(coin1), length(coin2))))

poiss <- summary(glm(heads ~ coin, data = DFrame, family = poisson))
p.poiss <- coef(poiss)[2,4]

data.frame(N1 = n1, p1 = p1,
N2 = n2, p2 = p2,
tosses = tosses,
binom_null = binom_null,
p_chisq = p.chisq,
p_poiss = p.poiss)
}

compareTests(5, 10, 1/2, 1/4, 100)

N1  p1 N2   p2 tosses binom_null      p_chisq      p_poiss
1  5 0.5 10 0.25    100        0.5 6.437112e-53 1.314629e-14


So, as you can see, you reach the same conclusion. But if you play around with sample sizes and null assumptions, you'll see that poisson regression is a good generalized solution.

rbind(compareTests(10, 10, 3/4, 1/4, 100, binom_null = .5),
compareTests(10, 10, 3/4, 1/4, 100))

N1   p1 N2   p2 tosses binom_null       p_chisq      p_poiss
1 10 0.75 10 0.25    100       0.50  6.386592e-01 3.244912e-50
2 10 0.75 10 0.25    100       0.75 1.866734e-148 1.387979e-54