# Chi2/ANOVA testing on sets of much different size

I am trying to decide on which stats test to use to determine if the number of males/females is higher on people who decided to buy a product versus those who didn't buy a product.

96000 people didn't buy the product. 4000 people bought the product. I have lists of buyers and non-buyers stating their gender (male/female)

so it goes [male, female, male, female, female, male, female, male, male, etc] for each list.

Null hypothesis: There is no difference in distribution in the ratio of males/females for those who bought the product versus those who didn't.

MY QUESTION: Because the sample sizes are so much different in size (4000 buyers and 96000 non buyers), can I still use a Chi2 test or do I have to use something else?

You give some totals for numbers of subjects and numbers of buyers, but you do not give particulars about the numbers of females and males, nor about the number of purchases by subjects of each gender. In the tests below, I use speculative counts for the missing ones. So your results will differ, but not the methods shown.

Test of equal proportions. Suppose 2100 female subjects out of 50,000 were buyers so that the estimated proportion of female buyers is $\hat p_f = 2100/50000 = 0.042.$ Also suppose 1900 male subjects out of 50,000 were buyers so that the estimated proportion of male buyers is $\hat p_m = 1900/50000 = 0.038.$ So you can frame this as a test of the equality $H_0: p_f = p_m$ of population proportions against inequality $H_a: p_f \ne p_m.$

In R, you can use the procedure prop.test to see whether there is a significant difference:

prop.test(c(2100,1900), c(50000,50000), corr=F)

2-sample test for equality of proportions without continuity correction

data:  c(2100, 1900) out of c(50000, 50000)
X-squared = 10.417, df = 1, p-value = 0.001249
alternative hypothesis: two.sided
95 percent confidence interval:
0.001571036 0.006428964
sample estimates:
prop 1 prop 2
0.042  0.038


The confidence interval is for the difference $p_f - p_m.$ The argument corr=F suppresses continuity correction, which I believe is not needed for such large sample sizes. The P-value 0.00125 indicates strong evidence that female and male purchase rates are different. (With continuity correction the P-value is 0.0013.)

Chi-squared test of independence. Alternatively, you could consider the $2 \times 2$ contingency table of purchasers and non-purchasers by gender:

f = c(2100, 47900);  m = c(1900, 48100);  DATA = cbind(m,f);  DATA
m     f
[1,]  1900  2100
[2,] 48100 47900


Then you could use the R procedure chisq.test to do a traditional chi-squared test of independence of the categories Gender and Purchase.

chisq.test(DATA, corr=F)

Pearson's Chi-squared test

data:  DATA
X-squared = 10.417, df = 1, p-value = 0.001249


Again here there is strong evidence that females and males have different purchase percentages. (With continuity correction the P-value is 0.0013.)

Fisher's Exact test. Another test for a $2 \times 2$ contingency table is Fisher's Exact test. It is based on the hypergeometric distribution.

fisher.test(DATA)

Fisher's Exact Test for Count Data

data:  DATA
p-value = 0.001318
alternative hypothesis: true odds ratio is not equal to 1
95 percent confidence interval:
0.8452425 0.9603891
sample estimates:
odds ratio
0.9010012


Some statisticians prefer to use Fisher's Exact Test only for one-sided alternatives. There are several methods for computing two-sided P-values. (See the R documentation for the method used in R. The confidence interval is for the odds ratio. R extends Fisher's exact test beyond two-by-two tables.)

Note: You asked about the extreme difference in the numbers of purchasers and non-purchasers. That is not problem with such a large number of subjects. For much smaller numbers of subjects, if the expected counts for any of the four cells is less than 5, then you should invoke the capability of the chisq.test in R to obtain a simulated P-value (with argument simulate.p=T) because the chi-squared approximation may not be valid.