Alternative test of chi-squared test for contingency table with categories not of interest

Question

I am trying to compare whether there is a significant difference in frequency of C1 and C2 between three samples. Every datapoint in each sample is labeled as C1, C2, C3, or C4. The contingency table is like this:

	C1	C2	C3	C4
Sample 1	count	count	count	count
Sample 2	count	count	count	count
Sample 3	count	count	count	count

The problem is I only care about C1 and C2 instead of C3 and C4. It does not make sense if I drop the count for C3 and C4 and do the chi-squared test. Is there any alternative to chi-squared test to help me understand whether there is any significant difference in frequency/percentage of C1 across samples, and that of C2 across samples?

Thank you.

Answer 1

I do not disagree with @whuber's suggestion to do a chi-squared test on a variant of the original data. However, it is often important to keep the overall purpose and design of a study in mind before looking at sub-sets of the data.

In particular, if the overall $3\times 4$ table were consistent with the null hypothesis of homogeneity or independence, then some people might say it is not proper to look for lack of homogeneity or independence in the submatrix consisting only of the first two columns out of context.

Consider the following fictitious counts:

s1 = c(22, 43, 10, 12)
s2 = c(45, 20, 11, 11)
s3 = c(25, 26, 21, 29)
TBL = rbind(s1,s2,s3);  TBL
    [,1] [,2] [,3] [,4]
s1   22   43   10   12
s2   45   20   11   11
s3   25   26   21   29

For this table the null hypothesis is rejected because of the very small P-value near $0.$

chisq.test(TBL, cor=FALSE)

        Pearson's Chi-squared test

data:  TBL
X-squared = 35.623, df = 6, p-value = 3.263e-06

The chi-squared statistic in the test above is the sum of the squares of the Pearson residuals shown below. Rows and columns having Pearson residuals with the larger absolute values draw attention to the cells of the table where observed and expected counts are in worst agreement. Here the first, second, and fourth columns seem of particular interest.

chisq.test(TBL, cor=FALSE)$resi
        [,1]      [,2]       [,3]      [,4]
s1 -1.317057  2.797384 -0.9018157 -1.097372
s2  2.946191 -1.537122 -0.6274802 -1.343922
s3 -1.512014 -1.169660  1.4193532  2.265786

Now we might look ad hoc at a chi-squared test on the sub-table consisting of the first two columns.

TBL.sm = TBL[ , 1:2];  TBL.sm
   [,1] [,2]
s1   22   43
s2   45   20
s3   25   26
chisq.test(TBL.sm)

        Pearson's Chi-squared test

data:  TBL.sm
X-squared = 16.374, df = 2, p-value = 0.0002782

We see that this sub-table also represents a significant effect--perhaps of practical interest, perhaps not. In your question you seem to think so.

Notes: (a) If the original table had more rows and columns we might find interesting tests for more than a few sub-tables. In that case, we should use Bonferroni or some other criterion for a stricter standard declaring significance in order to avoid 'false discovery' from multiple analyses on the same data.

(b) Familiarity with the purpose of the overall study behind TAB might lead to comparison of c1 with c3 or c1 with c4 as well as comparison of c1 with c2. perhaps even a comparison of c1 & c2 with c3 & c4.