2
$\begingroup$

I have this array:

name responsive not responsive
x 16 95
y 5 83

I am trying to compare the proportion of X_responsive with Y_responsive but the classic z-test of proportions can't be used here because my data is not normally distributed. I have used a chi-square test of proportions instead but I'm not sure what the output means in terms of my table.

import statsmodels.stats.proportion as ssp
(chi2, p, arr) = ssp.proportions_chisquare(count = df_obs.non_response, nobs = df_obs.sum(axis = 1))
print((chi2, p))
Chi2 = 3.96557520785049
p = 0.046439653061788856

I know it is significant but I'm not sure what is this is interpreted as.

$\endgroup$
12
  • 1
    $\begingroup$ You have binary variables, so you will never have normal data. Why do you think the z-test is inappropriate? // The $\chi^2$ test is a fine test to use, whether you think th z-test is appropriate or not. What confuses you about the output? $\endgroup$
    – Dave
    Commented Aug 4, 2021 at 13:56
  • $\begingroup$ What exactly the chi-square is revealing here. Is it just that there is a relationship between name and responsiveness? - if so, how can i determine the direction? whether is more responsive or y is more responsive? $\endgroup$
    – 1lapint3
    Commented Aug 4, 2021 at 14:01
  • 2
    $\begingroup$ Once they become binary, they cannot be normal. Normal distributions can take any real value, and binary distributions only take two values (quite a lot less than $\infty$). The bigger issue, though, is that you have destroyed information by binning your variables. This may warrant a separate question, but what were your starting $X$ and $Y$, and what question(s) do you want to use $X$ and $Y$ to answer? $\endgroup$
    – Dave
    Commented Aug 4, 2021 at 14:43
  • 1
    $\begingroup$ Because you used the $X$ and $Y$ values to determine the binning, the p-value no longer is correct. See stats.stackexchange.com/a/17148/919 for an example of what can go wrong. $\endgroup$
    – whuber
    Commented Aug 4, 2021 at 15:09
  • 1
    $\begingroup$ The two tailed z-test and the chi-squared are exactly the same test, with exactly the same assumptions. $\endgroup$
    – Glen_b
    Commented Aug 5, 2021 at 5:07

1 Answer 1

1
$\begingroup$

Comment: Illustration of @Glen-b's Comment about the equivalence of prop.test and chisq.test for your data.

The two procedures give exactly the same P-value:

prop.test(c(16,5), c(111,88))

        2-sample test for equality of proportions 
        with continuity correction

data:  c(16, 5) out of c(111, 88)
X-squared = 3.0944, df = 1, p-value = 0.07856
alternative hypothesis: two.sided
95 percent confidence interval:
 -0.004154855  0.178806780
sample estimates:
    prop 1     prop 2 
0.14414414 0.05681818 

TBL = rbind(c(16,5),c(95,83))
chisq.test(TBL)

        Pearson's Chi-squared test 
        with Yates' continuity correction

data:  TBL
X-squared = 3.0944, df = 1, p-value = 0.07856

They are also the same if the continuity correction is not used.

prop.test(c(16,5), c(111,88), cor=F)$p.val
[1] 0.04643965
chisq.test(TBL, cor=F)$p.val
[1] 0.04643965

However, there are several versions of the test for equality of two binomial proportions. Some versions use $H_0$ to argue for using a pooled sample proportion $\hat p = \frac{x_1+x_2}{n_1+n_2}$ to get the standard error for $\hat p_1 - \hat p_2$ and some use separate estimates $\hat p_i = x_i/n_i$ for this purpose. Also, various computer programs use different kinds of continuity corrections.

Also, if counts are too small for an accurate P-value in chisq.test, then R allows the option to simulate a more accurate P-value. (Simulation is not supported for prop.test.)

Finally, the Fisher Exact Test can give a different P-value than any of the above.

fisher.test(TBL)$p.val
[1] 0.06220786

Simulated p-values in chisq.test tend to be close to the Fisher p-value, especially if you use more than the default number of iterations to simulate.

Table with small counts:

TAB = rbind(c(40, 3), c(60, 7));  TAB
     [,1] [,2]
[1,]   40    3
[2,]   60    7

chisq.test(TAB)

        Pearson's Chi-squared test 
        with Yates' continuity correction

data:  TAB
X-squared = 0.077317, df = 1, p-value = 0.781

Warning message:
In chisq.test(TAB) : 
 Chi-squared approximation may be incorrect

chisq.test(TAB, sim=T)

        Pearson's Chi-squared test 
        with simulated p-value 
        (based on 2000 replicates)

data:  TAB
X-squared = 0.38181, df = NA, p-value = 0.7276

More iterations:

chisq.test(TAB, sim=T, B = 5000)

        Pearson's Chi-squared test 
        with simulated p-value 
        (based on 7000 replicates)

data:  TAB
X-squared = 0.38181, df = NA, p-value = 0.744

fisher.test(TAB)

        Fisher's Exact Test for Count Data

data:  TAB
p-value = 0.7374
alternative hypothesis: 
 true odds ratio is not equal to 1
95 percent confidence interval:
 0.3292961 9.8349592
sample estimates:
odds ratio 
  1.549608 
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.