Test of independence vs test of homogeneity

Question

I am teaching a basic statistics course and today I will cover the chi-squared test of independence for two categories and the test for homogeneity. These two scenarios are conceptually different, but can use the same test statistic and distribution. In a test of homogeneity, marginal totals for one of the categories are assumed to be part of the design itself -- they represent the number of subjects selected for each experimental group. But since the chi-squared test revolves around conditioning on all marginal totals, there are no mathematical consequences to distinguishing between tests of homogeneity and tests of independence with categorical data -- at least none when this test is used.

My question is the following: is there any school of statistical thought or statistical approach that would yield different analyses, depending on whether we are testing for independence (where all marginals are random variables) or a test of homogeneity (where one set of marginals are set by the design)?

In the continuous case, say where we observe $(X,Y)$ on the same subject, and test for independence, or observe $(X_1, X_2)$ in different populations and test if they come from the same distribution, the method is different (correlation analysis vs t-test). What if the categorical data came from discretized continuous variables? Should the tests of independence and homogeneity be indistinguishable?

Answer 1

There is a clear difference between the two problems if you model them in the Bayesian way. In some papers the first case (homogeneity) is called sampling with "one margin fixed" and the second case (independence) as "total table fixed". Have a look, for example, at Casella et al. (JASA 2009).
I am working on this topic but my paper - which also describe this distinction - is not out yet :)

Answer 2

You simply have to ask yourself, "How do I write the null hypothesis?". Consider a $2 \times k$ contingency table of frequencies of some behavior (y/n) among a number of $k$ groups. Treating the 1st group as referent, you have $k-1$ odds ratios ($\theta_i, i = 1, 2, \ldots, k-1$) that describe the association between frequency and group.

Under independence as with homogeneity, you assume that all odds-ratios are 1. That is, the likelihood of responding "yes" to the condition is equally likely irrespective of group assignment. If those assumptions fail, at least one group is different.

$\mathcal{H}_0(\mbox{homogeneity}): \sum_{i=1}^{k-1} |\theta_i| = 0$

$\mathcal{H}_0(\mbox{independence}): \sum_{i=1}^{k-1} |\theta_i| = 0$

And this test can be conducted with the Pearson Chi-square test using observed/expected frequencies, which is the score test for the logistic regression model adjusting for $k-1$ indicator variables for group membership. So structurally we may say that these tests are the same.

However, differences arise when we consider the nature of the grouping factor. In this sense, the contextual application of the test, or rather its name, is important. A group may be directly causal of an outcome, like the presence or absence of a gene or allele patterns of a trait in which case, when we reject the null we conclude that the outcome depends on the grouping factor in question.

On the other hand, when we test homogeneity, we exonerate ourselves of making any causal assumptions. Thus, when the "group" is a sophisticated construct like race (which causes and is caused by genetic, behavioral, and socioeconomic determinants) we can make conclusions like "racial-ethnic minorities experience housing disparities as evidenced by heterogeneity in neighborhood deprivation index". If someone countered such an argument by saying, "well that's because minorities achieve lower education, earn lower income, and gain less employment" you could say, "I didn't claim that their race caused these things, just simply that if you look at one's race, you can make predictions about their living condition."

In that way, tests of dependence are a special case of tests of homogeneity where the possible effect of lurking factors is of interest and should be handled in a stratified analysis. Using multivariate adjustment in the analogous logistic regression model achieves such a thing, and we may still say we are conducting a test of dependence, but not necessarily homogeneity.