Proportions Test for when one sample has lots of zeros

Question

I tried using chi-squared test, but it is giving me odd results,

For example:

Let's say that I am testing whether behavior between male and female users is the same. Males only bought 3 units of the first item, while females bought 5 units of every item.

males <- matrix(0,100)
females <- matrix(5,100)

males[1,1] = 3

males_and_females = cbind(males, females)

print(prop.test(males_and_females))

Result:

data:  males_and_females
X-squared = 186.7387, df = 99, p-value = 2.376e-07

It is telling that we can reject the null hypothesis that two samples proportions are the same. To me it is very strange that the test is telling us something with high confidence when there are only 3 observations for male users. This certainly feels wrong to me. I am probably using the test for what it is not meant for. I went over the implementation of chi-squared test, and I see that the high chi-squared comes from dividing by tiny expected values associated with zeros in matrix a.

It looks like R is suspicious of what I am asking it to do:

Warning message:
In prop.test(males_and_females) :
  Chi-squared approximation may be incorrect

My question: is there a version of adjusted chi-squared test that addresses the above issues? Or is there another test that handles this better? Or any other suggestions on how to test whether the proportions of two samples are the same when one sample has lots of zeros.

Edit:

print(chisq.test(males_and_females, simulate.p.value=TRUE, B=1e5))

Rusult:

    Pearson's Chi-squared test with simulated p-value (based on 1e+05 replicates)

data:  males_and_females
X-squared = 186.7387, df = NA, p-value = 7e-05

Answer 1

The answer is correct, please read the Details section in ?prop.test. You have more than two groups (you have, in fact, 100 groups). In 99 of those groups, you had 0 successes and 5 failures (P=0). In 1 of those groups, you had 3 successes and 5 failures (P=3/8). The hypothesis being tested is that all of the groups share the same probability for success... and this hypothesis is (rightly) rejected.

Try typing just prop.test(a_and_b) (i.e. without print), and you'll see all the groups being treated as independent samples.

If you reduce the number of P=0 groups, you'll start to see the test-statistic drop (and similarly the p-value rise):

> prop.test(a_and_b[1:10,])

    10-sample test for equality of proportions without continuity
    correction

data:  a_and_b[1:10, ]
X-squared = 17.8875, df = 9, p-value = 0.0365
alternative hypothesis: two.sided
sample estimates:
 prop 1  prop 2  prop 3  prop 4  prop 5  prop 6  prop 7  prop 8  prop 9 prop 10 
  0.375   0.000   0.000   0.000   0.000   0.000   0.000   0.000   0.000   0.000