# Interpretation of chi-squared on proportion

I have this array:

name responsive not responsive
x 16 95
y 5 83

I am trying to compare the proportion of X_responsive with Y_responsive but the classic z-test of proportions can't be used here because my data is not normally distributed. I have used a chi-square test of proportions instead but I'm not sure what the output means in terms of my table.

import statsmodels.stats.proportion as ssp
(chi2, p, arr) = ssp.proportions_chisquare(count = df_obs.non_response, nobs = df_obs.sum(axis = 1))
print((chi2, p))

Chi2 = 3.96557520785049
p = 0.046439653061788856


I know it is significant but I'm not sure what is this is interpreted as.

• You have binary variables, so you will never have normal data. Why do you think the z-test is inappropriate? // The $\chi^2$ test is a fine test to use, whether you think th z-test is appropriate or not. What confuses you about the output?
– Dave
Commented Aug 4, 2021 at 13:56
• What exactly the chi-square is revealing here. Is it just that there is a relationship between name and responsiveness? - if so, how can i determine the direction? whether is more responsive or y is more responsive? Commented Aug 4, 2021 at 14:01
• Once they become binary, they cannot be normal. Normal distributions can take any real value, and binary distributions only take two values (quite a lot less than $\infty$). The bigger issue, though, is that you have destroyed information by binning your variables. This may warrant a separate question, but what were your starting $X$ and $Y$, and what question(s) do you want to use $X$ and $Y$ to answer?
– Dave
Commented Aug 4, 2021 at 14:43
• Because you used the $X$ and $Y$ values to determine the binning, the p-value no longer is correct. See stats.stackexchange.com/a/17148/919 for an example of what can go wrong.
– whuber
Commented Aug 4, 2021 at 15:09
• The two tailed z-test and the chi-squared are exactly the same test, with exactly the same assumptions. Commented Aug 5, 2021 at 5:07

Comment: Illustration of @Glen-b's Comment about the equivalence of prop.test and chisq.test for your data.

The two procedures give exactly the same P-value:

prop.test(c(16,5), c(111,88))

2-sample test for equality of proportions
with continuity correction

data:  c(16, 5) out of c(111, 88)
X-squared = 3.0944, df = 1, p-value = 0.07856
alternative hypothesis: two.sided
95 percent confidence interval:
-0.004154855  0.178806780
sample estimates:
prop 1     prop 2
0.14414414 0.05681818

TBL = rbind(c(16,5),c(95,83))
chisq.test(TBL)

Pearson's Chi-squared test
with Yates' continuity correction

data:  TBL
X-squared = 3.0944, df = 1, p-value = 0.07856


They are also the same if the continuity correction is not used.

prop.test(c(16,5), c(111,88), cor=F)$$p.val [1] 0.04643965 chisq.test(TBL, cor=F)$$p.val
[1] 0.04643965


However, there are several versions of the test for equality of two binomial proportions. Some versions use $$H_0$$ to argue for using a pooled sample proportion $$\hat p = \frac{x_1+x_2}{n_1+n_2}$$ to get the standard error for $$\hat p_1 - \hat p_2$$ and some use separate estimates $$\hat p_i = x_i/n_i$$ for this purpose. Also, various computer programs use different kinds of continuity corrections.

Also, if counts are too small for an accurate P-value in chisq.test, then R allows the option to simulate a more accurate P-value. (Simulation is not supported for prop.test.)

Finally, the Fisher Exact Test can give a different P-value than any of the above.

fisher.test(TBL)\$p.val
[1] 0.06220786


Simulated p-values in chisq.test tend to be close to the Fisher p-value, especially if you use more than the default number of iterations to simulate.

Table with small counts:

TAB = rbind(c(40, 3), c(60, 7));  TAB
[,1] [,2]
[1,]   40    3
[2,]   60    7

chisq.test(TAB)

Pearson's Chi-squared test
with Yates' continuity correction

data:  TAB
X-squared = 0.077317, df = 1, p-value = 0.781

Warning message:
In chisq.test(TAB) :
Chi-squared approximation may be incorrect

chisq.test(TAB, sim=T)

Pearson's Chi-squared test
with simulated p-value
(based on 2000 replicates)

data:  TAB
X-squared = 0.38181, df = NA, p-value = 0.7276


More iterations:

chisq.test(TAB, sim=T, B = 5000)

Pearson's Chi-squared test
with simulated p-value
(based on 7000 replicates)

data:  TAB
X-squared = 0.38181, df = NA, p-value = 0.744

fisher.test(TAB)

Fisher's Exact Test for Count Data

data:  TAB
p-value = 0.7374
alternative hypothesis:
true odds ratio is not equal to 1
95 percent confidence interval:
0.3292961 9.8349592
sample estimates:
odds ratio
1.549608