I tried using chi-squared test, but it is giving me odd results,
Let's say that I am testing whether behavior between male and female users is the same. Males only bought 3 units of the first item, while females bought 5 units of every item.
males <- matrix(0,100) females <- matrix(5,100) males[1,1] = 3 males_and_females = cbind(males, females) print(prop.test(males_and_females))
data: males_and_females X-squared = 186.7387, df = 99, p-value = 2.376e-07
It is telling that we can reject the null hypothesis that two samples proportions are the same. To me it is very strange that the test is telling us something with high confidence when there are only 3 observations for male users. This certainly feels wrong to me. I am probably using the test for what it is not meant for. I went over the implementation of chi-squared test, and I see that the high chi-squared comes from dividing by tiny expected values associated with zeros in matrix
It looks like
R is suspicious of what I am asking it to do:
Warning message: In prop.test(males_and_females) : Chi-squared approximation may be incorrect
My question: is there a version of adjusted chi-squared test that addresses the above issues? Or is there another test that handles this better? Or any other suggestions on how to test whether the proportions of two samples are the same when one sample has lots of zeros.
print(chisq.test(males_and_females, simulate.p.value=TRUE, B=1e5))
Pearson's Chi-squared test with simulated p-value (based on 1e+05 replicates) data: males_and_females X-squared = 186.7387, df = NA, p-value = 7e-05