1
$\begingroup$

First I calculated the chi-squared p-value with

data = pd.crosstab(df['variable1'],df['variable2'])
chi2, p, dof, expected = chi2_contingency(data)

print("Chi-squared statistic: ", chi2)
print("p-value: ", p)

Chi-squared statistic: 70.23601804402738; p-value: 2.893285699471121e-10; Degree of Freedom: 12

Then I tried to calculate it manually using the following code

data.values
observed_values = data.values
val = stats.chi2_contingency(data)
chisq_stat, pvalue, df, expected = chi2_contingency(observed_values)
expected_values = val[3]
no_of_rows=len(data.iloc[0:4,0])
no_of_columns=len(data.iloc[0,0:5])
ddof=(no_of_rows-1)*(no_of_columns-1)
print("Degree of Freedom:-",ddof)
alpha = 0.05
chi_square=sum([(o-e)**2/e for o,e in zip(observed_values,expected_values)])
chi_square_statistic=chi_square[0]+chi_square[1]
print(chi_square)
print("chi-square statistic:-",chi_square_statistic)
critical_value=chi2.ppf(q=1-alpha,df=ddof)
print('critical_value:',critical_value)
p_value=1-chi2.cdf(x=chi_square_statistic,df=ddof)
print('p-value:',p_value)
print('Significance level: ',alpha)
print('Degree of Freedom: ',ddof)

These are the results from the above code

chi-square statistic:- 11.94630668094643; critical_value: 21.02606981748307; p-value: 0.45000142553796574; Significance level: 0.05; Degree of Freedom: 12

Where I am doing wrong?

$\endgroup$

1 Answer 1

0
$\begingroup$

Please add a reproducible example for the data you are using.

The python function employs a continuity correction (adding 0.5 to each observation towards the expected). Try running it with:

scipy.stats.chi2_contingency(observed, correction=False)

It should return the correct answer. You can read more about it in the function documentation - https://docs.scipy.org/doc/scipy/reference/generated/scipy.stats.chi2_contingency.html

$\endgroup$
1
  • $\begingroup$ I think in manual method chi_square_statistic=chi_square[0]+chi_square[1] is not adding statistics of all the 5 columns...Only adding adding two. I just replaced this line with chi_square_statistic=chi_square.sum...now both methods are producing same results... $\endgroup$ Jan 24, 2023 at 9:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.