# Statsmodels tests for proportions give different results than other software [closed]

An example from Primer of Biostatistics by Glantz: 8 patients out of 61 died while undergoing heart surgery with one anesthetic. With another anesthetic, 10 patients died out of 67.

If we use R, chisq.tst and prop.test give the same p-value of 0.9683:

mort <- matrix(c(8, 10, 53, 57), nrow = 2)
rownames(mort) <- c('Galotan', 'Morphine')
colnames(mort) <- c('Survived', 'Died')
chisq.test(mort)
prop.test(mort)


Same result from chi2_contingency:

df = pd.DataFrame({'Died': [8, 10], 'Survived': [53, 57]})
scipy.stats.chi2_contingency(df)


Statsmodels has two functions apparently for the same purpose. Both give the same p-value which is different from p-value from R and scipy, although the expected table is exactly the same as in scipy.

df['Total'] = df.Died + df.Survived
statsmodels.stats.proportion.proportions_chisquare(df.Died, df.Total)
statsmodels.stats.proportion.proportions_ztest(df.Died, df.Total, value=None, alternative='two-sided', prop_var=False)

• Reading the help pages will enable you to identify the differences.
– whuber
Sep 14 at 15:18

Unit test in statsmodels are against R's prop.test without Yates continuity correction.

Using continuity correction makes the test similarly conservative as Fisher's exact test and under rejects on average. See, for example Agresti Coull

Agresti, Alan, and Brent A. Coull. "Approximate is better than “exact” for interval estimation of binomial proportions." The American Statistician 52, no. 2 (1998): 119-126.

> prop.test(mort, correct=FALSE)

2-sample test for equality of proportions without continuity
correction

data:  mort
X-squared = 0.086617, df = 1, p-value = 0.7685
alternative hypothesis: two.sided
95 percent confidence interval:
-0.1383397  0.1021273
sample estimates:
prop 1    prop 2
0.1311475 0.1492537


Statsmodels has now specific functions for comparing proportions from two independent samples. Those implement several methods that are recommended in the literature. Those methods do not guarantee that the size is maintained in all cases, but maintain it on average and have higher power.
https://www.statsmodels.org/dev/generated/statsmodels.stats.proportion.test_proportions_2indep.html

For example, using score test that difference between two independent proportions is zero. In this example p-value of the score test is close to the pvalue of the chisquare test without continuity correction.

import statsmodels.stats.proportion as prop
prop.test_proportions_2indep(8, 61, 10, 67, value=None,
method="score", compare='diff',
alternative='two-sided', correction=True,
return_results=True)
<class 'statsmodels.stats.base.HolderTuple'>
statistic = -0.2931559182830287
pvalue = 0.7694029766512749
compare = 'diff'
method = 'score'
variance = 0.0038146687499638764
alternative = 'two-sided'
prop1_null = 0.140625
prop2_null = 0.140625
tuple = (-0.2931559182830287, 0.7694029766512749)
diff = -0.018106190359677005
ratio = 0.8786885245901641
odds_ratio = 0.8603773584905663
value = 0


Besides the hypothesis tests for two independent sample, statsmodels also has confidence intervals, power function and equivalence testing (TOST) for this case.

One of the references that compares methods for confidence intervals that was used to select which methods to provide in statsmodels is

Fagerland, M.W., Lydersen, S. and Laake, P., 2015. Recommended confidence intervals for two independent binomial proportions. Statistical methods in medical research, 24(2), pp.224-254.

• Thank you! confint_proportions_2indep in statsmodels also gives the same confidence interval as prop.test without continuity correction. Sep 13 at 19:47