Which test to use to compare 2 %'s of dependent values?

Question

For this example, there are 100k people who have purchased a backpack. Out of the 100k that did purchase a backpack, 75% of the backpack buyers are female. 3.9 million people did not purchase the backpack. Out of the 3.9M that did not purchase the backpack, 76% are female.

I am trying to find which test to use to figure out if men are more likely to shop for backpacks.

So in summary: Group A:

• 75% of buyers are female

• 100k total buyers of backpacks

Group B:

• 76% of buyers are female
• 3.9M non-buyers of backpacks

Is there a significant difference between the 2 groups in terms of if men shop for backpacks more than females?

Answer 1

You can use a chi-squared ($\chi^2$) test. I simulate what your data would look like below, but with less N:

# make data frame
dat <- data.frame(
  group = c(rep("A", 100), rep("B", 3900)),
  gender = c(rep("F", 75), rep("M", 25), rep("F",  2964), rep("M", 936))
)

# randomly re-arrange so top 10 shows variability
set.seed(1839)
dat  <- dat[sample(1:nrow(dat), nrow(dat)), ]

A sample of the data would look like this:

# show example of data
head(dat)

     group gender
2091     B      F
182      B      F
3887     B      M
2217     B      F
758      B      F
2136     B      F

From these data, you can make a frequency table:

# show table
table(dat$group, dat$gender)

And run a chi-squared test:

# run chisq test
chisq.test(dat$group, dat$gender)

Which, in this case, suggests that the 75% female rate is not statistically significantly different from the 76% female rate:

    Pearson's Chi-squared test with Yates' continuity correction

data:  dat$group and dat$gender
X-squared = 0.012678, df = 1, p-value = 0.9104

You can also feed in a frequency table from your data:

# frequency table of entire data
freq <- matrix(
  c(100000 * .75, 100000 * .25, 
    3900000 * .76, 3900000 * .24), 
nrow  = 2)

# do chisq test
chisq.test(freq)

Which shows a significant difference:

Pearson's Chi-squared test with Yates' continuity correction

data:  freq
X-squared = 53.361, df = 1, p-value = 2.775e-13

It is worth noting that, in such large samples, even very trivially small differences can return a result that is statistically significant. It is up to you on the interpretation end to decide whether or not that 1 point difference is actionable.