What is the $\chi^2$ contingency test really showing?

Question

I have a friend who used this function to compute a chi-squared contingency table in a paper they are writing, and I don't think I fully understand what the results of this test are.

Function docstring in case the link breaks:

Chi-square test of independence of variables in a contingency table.

This function computes the chi-square statistic and p-value for the hypothesis test of independence of the observed frequencies in the contingency table [1] observed. The expected frequencies are computed based on the marginal sums under the assumption of independence; see scipy.stats.contingency.expected_freq. The number of degrees of freedom is (expressed using numpy functions and attributes):

dof = observed.size - sum(observed.shape) + observed.ndim - 1

The null hypothesis, as I understand it, is that the predictor variables are independent of the groups. More accurately, it is saying that the same distribution generated the data for all the groups.
What I don't understand is the alternative hypothesis. Is this saying that none of the variables are associated with the groups, or that they are collectively independent from the groups?

Answer 1

Let's start with a small example. Say we have two groups, smokers and non-smokers, and we want to know the proportion of those who developed lung cancer.

The data look like

m = np.array([[10, 29,], [90, 71]])

10 of the 100 non-smokers developed lung cancer, and 29 out of 100 of the smokers developed lung cancer.

The null hypothesis for the test can be restated as follows: The frequency of deaths within each group (smokers and non-smokers) is the same.

The alternative hypothesis sis that the frequencies are different within groups (or more precisely, that the frequency of deaths within each group is not the same). That can be interpreted in a bunch of ways, but what it really means is that there is at least once group with a different frequency. For two groups, that boils down to a difference between groups, but for more than 2 groups it means at least one is different.

Let's run the test.

from scipy.stats import chi2_contingency
import numpy as np

m = np.array([[10, 29,], [90, 71]])

Xi2, pval, df, expected = chi2_contingency(m)

print(pval)
>>>0.0013158811321789705

The p value is smaller than our nominal value (usually 0.05). This means that the probability of observing data at least as extreme as the ones we have observed is quite unlikely assuming the two did in reality have the same frequency of lung cancer. We would then conclude that the frequency of lung cancer between the two groups is different.