Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It only takes a minute to sign up.
Sign up to join this community
I sometimes use the chi-squared test to test the independence of categorical variables. I know how to do the test but don't know the underlying concepts and details.
So, I was wondering about the following:
Instead of asking why the chi-squared test assesses independence, think about how you would test the association of two categorical variables against the null hypothesis of independence. For the sake of simplicity, consider two variables / properties A and B, each with two levels, "yes" and "no". A set of N units each has some combination of those properties. For example, A could be lung cancer, B could be exposure to radon, and the units could be people / patients. The numbers might look like this:
table = as.table(rbind(c(1,2),
c(3,4) ))
names(dimnames(table)) = c("lung.cancer", "radon")
rownames(table) = c("yes", "no")
colnames(table) = c("no", "yes")
table
# radon
# lung.cancer no yes
# yes 1 2
# no 3 4
Now, we want to test these variables for independence; what do we mean by "independence"? We aren't interested in testing if any of the proportions are any particular value. We want to know if being in a particular column is associated with being in a particular row. If the rows and columns are independent, knowledge of which column a patient is in provides no information about which row they are in.
Our total number of patients in this study is $N = 10$. How would those observations be distributed under the null (i.e., if the variables were independent)? Well, the probability of having been exposed to radon is $Pr(R = {\rm yes})$, and the probability of having lung cancer is $Pr(LC = {\rm yes})$, so from basic probability we know that the probability of having been exposed to radon and having lung cancer is $Pr(R = {\rm yes})Pr(LC = {\rm yes})$, and the probability of not having been exposed to radon and having lung cancer is $(1-Pr(R = {\rm yes}))Pr(LC = {\rm yes})$, etc. To get the expected count, we can multiply those probabilities by N. A problem here is that we don't know the probabilities of people being exposed to radon or of having lung cancer. We can estimate those probabilities from our data as the number who had been exposed to radon / have lung cancer divided by the total. Thus, we can estimate the component probabilities and ultimately the expected counts under the null.
ex = chisq.test(table)$expected; ex
# radon
# lung.cancer no yes
# yes 1.2 1.8
# no 2.8 4.2
When we try to compare the counts we observed to the expected counts we just calculated, we will run into two problems. The first is that the differences will sum to $0$:
table-ex
# radon
# lung.cancer no yes
# yes -0.2 0.2
# no 0.2 -0.2
To address that problem, we can square the differences and sum the squares. The second problem is that the magnitude of the differences (or their squares) will tend to increase as a function of the expected count. We can address that by dividing each squared difference by the expected count.
So now we have a perfectly simple, intuitive way to test if the variables are independent. Notice, however, that we have just re-created the Pearson's chi-squared test statistic:
$$
\chi^2 = \frac{\sum (O - E)^2}{E}
$$
All we need to know now is how the test statistic should be distributed under the null. It turns out it is distributed as a chi-squared on $(r-1)(c-1)$ degrees of freedom, where $r$ is the number of rows and $c$ is the number of columns. (Actually, it isn't quite because the chi-squared distribution can take any non-negative real value, whereas the test statistic can only a specific set of values when $N$ is finite; this fact leads to some additional complications that I won't discuss here.)
The above should cover questions 1 and 2. For question 3, it may help you to read this excellent CV thread: What is the meaning of p values and t values in statistical tests? To answer your explicit question about $.05$ and $.01$, those numbers are completely arbitrary and come from tradition. As far as I know, their exact origins are shrouded in the mists of time.
$\chi_1^2$ is the distribution of a random variable $X^2$, where $X \sim N(0,1)$.
In the test of independence of two categorical variables, the actual counts are subtracted from the expected counts, and normalized by dividing by the expected counts. Squaring is extremely useful, because we are not interested in the direction of the differences (positive or negative with respect to the expected counts), but rather the magnitude in absolute values. Absolute values are awkward, and squaring fulfills the same role, plus it fits into the $\chi^2$ just right.
Next we add these square normalized differences from each cell to produce the $\chi^2$ statistic: $\small\displaystyle \sum_{i=1}^n\frac{(\text{Observed - Expected})^2}{\text{Expected}}$. This, under some minimal conditions will follow a $\chi^2$ distribution. The $\small \text{deg. of freedom}$ will be calculated as $(\small\text{no.rows - 1})\times(\text{no.cols - 1})$. We are running an "omnibus" test to assess the overall deviation from what is expected by adding together the squared differences in each cell, and keeping track of the degrees of freedom.
In the test of independence, the margins are considered fixed and the cell expected counts are conditioned on both margins.
Alan Agresti has a great example on page 65 of Categorical Data Analysis (Second Edition) based on sampling couples from the population, and assessing whether the sexual satisfaction of the wife is independent of the sexual satisfaction of the husband - intuition would reason that happy couples would reflect in similar assessment of satisfaction for both husband and wife. Each couple is an observational unit, and two categorical variables are recorded for each unit. These concepts are a bit subtle, but important in differentiating the test of independence from the test of homogeneity. I tried summarizing them here.
The tabulated data are as follows:
husband
wife Never Sometimes Often Always Sum
Never 7 7 2 3 19
Sometimes 2 8 3 7 20
Often 1 5 4 9 19
Always 2 8 9 14 33
Sum 12 28 18 33 91
Running a chi-square test of independence will yield:
chisq.test(sex_satis, correct = T)
## Pearson's Chi-squared test
##
## data: sex_satis
## X-squared = 16.955, df = 9, p-value = 0.04942
Which means that the *probability of encountering a test statistic more extreme than $\small 16.95$ under the null hypothesis (i.e. independence between satisfaction of husbands and wives) is less than $5\%$ (exactly $\small 0.049$), and for many studies this is good enough to reject $\text{H}_0$- indeed, in this case, the intuition of common sense is reaffirmed by data.
Now this should answer the question but I want to tie up the point about conditioning on the margins. Because we don't know the proportions in the population, we estimate them based on the sample, and we figure that under the null hypothesis of independence, the expected value of “never/never” is the probability that the wife “never” experiences satisfaction ($19/91$) times (independent of) the probability that the husband “never” experiences pleasure ($12/91$). Based on this resultant probability the expected frequency in the cell will be found by multiplying by the total number of cases ($91$).
The threshold of p value is usually set at 0.05, which can be traced back to the beginning of statistics. If you want to be more conservative, you can of course use p<0.01. It just shows how unlikely the null hypothesis of independence is, therefore leading to a reasonable rejection of the null hypothesis.
An example below is borrowed from https://www.icalcu.com/stat/chisqtest.html
"In a final exam, 5 girls and 6 boys got A, 7 girls and 8 boys got A-, 8 girls and 7 boys got B, 6 girls and 5 boys got B-. In order to test whether gender and final exam score are independent, we would input "5 7 8 6" in the first row above for girls, and "6 8 7 5" in the second row above for boys. We get a chi-square value of 0.3151515 with 3 degrees of freedom, and the associated p value is at 0.9571542. As p>0.05, we can not reject the null hypothesis that gender and final exam score are independent."
