Chi Square test of Independence - Why Chi sq distribution?

Question

I read the theory of Chi square test of independence and I am comfortable with the idea of expected counts and the reasoning behind calculating them the way they are. But what bothers me is the selection of Chi square distribution for getting the p-value. Why do we use the Chi square distribution as the distribution to measure against? And how does it work for categorical variables?

Answer 1

The counts random variables can be thought as a sum of independent and identically distributed Bernoulli random variables. Take an example, suppose you have a categorical observations which tells you gender of a person. Thus from a sample on $n$ observations, some of them will be male and some of them not. If all observations are independent identically distributed then you can consider the random variable Male to be a binomial variable, since a binomial is defined as a sum of i.i.d. Bernoulli variables. So we established that a count has a Binomial distribution with $p$ being the probability from the Bernoulli variable and $n$ the number of observations.

The next immediate step is to consider the fact that the expected value of a Binomial variable can be approximated by a normal distribution. A simple rationale for that is the central limit theorem, which states that for any sample of independent and identically distributed variables, the sample mean is normally distributed. Notice, however that in the chi-square test, the expected counts are sum of squares of binomial variables. Since binomial can be approximated by normal distribution, we can state that the expected counts are sum of squares of normal i.i.d. variables.

The last step is to follow the definition of the chi square distribution. So the chi square distribution is defined as a sum of squared independent and identically distributed standard normal distributions. Since your counts follows normal distribution, those distributions can be standardized by factoring out variance and mean and become standard normal, and consequently their squares follows a chi-square distribution.

Regarding the degree of freedom the discussion is more involved and requires some formalism to show how they are derived, which I do not include here, since my purpose was to provide you some intuitions only.