Based on this answer, Python requires expected values in a chi square test to be absolute frequencies. Consider the following in Python:

import numpy
import scipy.stats
# chisquare function requires (observed, expected)
scipy.stats.chisquare(numpy.array([0,0,23,0]), numpy.array([1,1,1794,1]))
(1751.2948717948718, 0.0)

results in a p-value of 0 (whatever that means).

The same calculation in R, which requires that the expected values be proprotions:

chisq.test(c(0, 0, 23, 0), p=c(1/1797,1/1797,1794/1797, 1/1797))

        Chi-squared test for given probabilities

data:  c(0, 0, 23, 0)
X-squared = 0.0385, df = 3, p-value = 0.998

resulting in a p-value of 0.998.

Which is correct?


These two seem to be testing different things. The Python code looks like it is a two way chi square test (but a p value of 0 makes no sense here), while the R code is one way. I am not sure which you want.

To do the two way test in R use

x1 <- c(0, 0, 23, 0)
x2 <- c(1, 1, 1794, 1)
chisq.test(x1, x2)

Which gives a p value of 0.5.

However, since a lot of the expected values are less than 5, Rcorrectly gives a warning. You can simulate using

chisq.test(x1, x2, simulate = TRUE)

which gives a p of 0.25

Your code also gives a warning, but this

chisq.test(c(0, 0, 23, 0),
           p=c(1/1797,1/1797,1794/1797, 1/1797),
           simulate = TRUE)

gives a p of 1.

This certainly makes sense.

I don't have Python so I can't say for sure what is going on there.

A two way chi square tests whether two categorical variables are associated with each other; a one way tests whether one categorical variable is distributed equal to a certain set of proportions.

  • $\begingroup$ I'm trying to determine if the frequency of the categories of the two sets of observed values differ from each other. I'm doing two tests: one using one dataset as the expected, then the other way round. $\endgroup$ – SabreWolfy Oct 26 '13 at 13:09
  • $\begingroup$ More in my question here: stats.stackexchange.com/questions/73829/… $\endgroup$ – SabreWolfy Oct 26 '13 at 13:24
  • 1
    $\begingroup$ Then you want a two way test, and you only need one of them. $\endgroup$ – Peter Flom Oct 26 '13 at 13:40
  • 2
    $\begingroup$ FWIW: I've determined that chisquare in Python is a one-way test that the categorical data has the given frequencies, and is invalid when observed or expected frequencies are below 5 (as is typically the case). In this case, it makes sense to use R rather than Python. $\endgroup$ – SabreWolfy Oct 26 '13 at 13:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.