# Chi-Square Test Confidence

I'm considering apply chi-square test to understand if two categorical variables are related. I made some search on web and saw some examples in which chi-square statistic is calculated by using some library or tool and then a critical value is calculated from the chi-square distribution. Then simply if the calculated chi-square statistic below the critical value they concluded that there is no relationship between categorical variables or vice versa. I wonder difference between calculated chi-square statistic and critical value is also significant? I mean higher the chi-square test statistic does it also mean higher the relationship between categorical variables?

I believe you will be better able to use your intuition if you look at P-values, instead of values of the chi-squared test statistic.

Test statistic. The chi-squared statistic $$Q$$ used to test for independence in a two-way contingency table can be considered as a numerical measure of how badly the actual counts match the expected counts derived by assuming independence. Larger values of the statistic indicate greater association between the two categorical variables.

However, it is difficult to develop good intuition about the practical meaning of particular values of $$Q.$$ For example $$Q=0,$$ which happens extremely rarely for honest data, would mean the data are a perfect match to independence. However in any particular situation it would be hard to guess what to make of $$Q = 10.$$

So it is better to consider how likely it would be for various values of $$Q$$ to occur if the variables are independent.

Critical value. The critical value $$c$$ for a test at the 5% level is used as you say. Reject the null hypothesis of independence of $$Q > c.$$ The critical value depends on the number of rows and columns of the table.

In a table with two rows and three columns, the critical value for a test at the 5% level. The degrees of freedom are $$df = (2-1)(3-1) = 2$$ and a chi-squared distribution with $$df=2$$ has 5% of its probability to the right of the critical value is $$c = 5.99,$$ as shown in the computation below in R or as can be found in printed tables of chi-squared distributions. (There is no use trying to guess the critical value $$c$$ because chi-squared distributions with different degrees of freedom can have very different shapes.)

qchisq(.95, 2)
[1] 5.991465


In the graph below, the position of the vertical dotted line is $$c = 5.99.$$ The area under the density curve to the right of that line is 5% of the total area under the curve.

Specific example (data consistent with independence): Consider the fake data in the contingency table below: The first row shows increasing counts from left to right and the second row does not. So the row and column categorical variables may not be independent.

MAT = matrix(c(40,70,90, 20,20,20), byrow=T, nrow=2); MAT
MAT
[,1] [,2] [,3]
[1,]   40   70   90
[2,]   20   20   20


The chi-squared statistic for this table is $$Q = 5.08 < 5.99,$$ so counts in this table do not significantly contradict independence.

out =chisq.test(MAT, cor=F);  out

Pearson's Chi-squared test

data:  MAT
X-squared = 5.0774, df = 2, p-value = 0.07897


Computed under the assumption of independence, the corresponding expected counts are as follows--different from the data, but not convincingly so.

out\$exp
[,1]     [,2]     [,3]
[1,] 46.15385 69.23077 84.61538
[2,] 13.84615 20.76923 25.38462


In R, the output from the test shows the P-value to be 0.0789. This is the probability that independent counts would give a $$Q$$ that exceeds our $$Q = 5.078.$$ Because the P-value exceeds 5%, we cannot reject the null hypothesis of independence.

In the figure below, the heavy black vertical line shows the position of $$Q.$$ The percentage of the area under the density curve to the right of $$Q$$ is the P-value, nearly 8%.

Conclusion. My major point here is that it's possible to make intuitive sense of P-values. They are on the same (probability) scale no matter the number of cells in the data table or the relative sizes of the counts.

A very small P-value (much below 5%) means it is very unlikely for data in the table to have occurred if categories were truly independent.

Another example, in which independence is rejected: By contrast, the slightly different data table below leads to a larger $$Q$$ and a P-value 0.0013 (much below 5%), indicating that its counts are pretty clearly not consistent with independence.

MAT = matrix(c(40,70,90, 20,20,10), byrow=T, nrow=2); MAT
[,1] [,2] [,3]
[1,]   40   70   90
[2,]   20   20   10
chisq.test(MAT, cor=F)

Pearson's Chi-squared test

data:  MAT
X-squared = 13.194, df = 2, p-value = 0.001364


• Thank you so much. It's a quite informative reply. Apr 6, 2020 at 7:06