# I study subpopulation so huge that chi-squares test insists it is not representative. How do I get confidence intervals for my population?

I want to get results about population A. But all I can study is subpopulation C of B, where B is a subpopulation of A (that is, C $\subset$ B $\subset$ A). Each member of A is classified into one of the groups G1, G2, G3 and G4. Selection from A into B is not random, but selection from B into C is random, although some members of B have better chances to be selected into C.

To test if subpopulation C is representative of A, I constructed the "contingency" table ...

      A             B          C        Total?
G1     9817678      621718     90196    10,529,592
G2     1280567      191298     30799    1,502,664
G3     426856       79707      11000    517,563
G4     9817678      701425     87996    10,607,099
Total  21,342,779   1,594,148  219,991  23,156,918


... ignoring my doubts about column "Total?". For example, grand total 23,156,918 is pretty meaningless, since there are only 21,342,779 physically distinct members.

And, of course, for such huge populations the chi-squares test insists that B is not representative for A, and C is not representative for B.

 d <- data.frame(
cbind(
A = c(9817678, 1280567, 426856, 9817678),
B = c(621718, 191298, 79707, 701425),
C = c(90196, 30799, 11000, 87996)),
row.names = c('G1', 'G2', 'G3', 'G4'))

chisq.test(d[,1:2])
#
#   Pearson's Chi-squared test
#
#data:  d[, 1:2]
#X-squared = 160929.2, df = 3, p-value < 2.2e-16

chisq.test(d[,2:3])
#
#   Pearson's Chi-squared test
#
#data:  d[, 2:3]
#X-squared = 1539.55, df = 3, p-value < 2.2e-16


Am I applying the wrong statistical test here?

Now, if I would randomly sample 100 members from A, B and C, and get the "average" numbers for G1 - G4, the chi-squared will be pretty confident that C is representative of B:

 d.r100 <- sapply(d, function(x) round(100*x/sum(x)))
d.r100
#      A  B  C
#[1,] 46 39 41
#[2,]  6 12 14
#[3,]  2  5  5
#[4,] 46 44 40

chisq.test(d.r100[,2:3])
#
#   Pearson's Chi-squared test
#
#data:  d.r100[, 2:3]
#X-squared = 0.3943, df = 3, p-value = 0.9414


It feels wrong that the more data I process, the worse my representativeness becomes.

After all, if studying "subpopulation C" will fetch me the answer for "population A" plus-minus mere 2-3%, it will be fine. But how do I estimate my confidence intervals for "A" if all I can study is "C"?

• Daniil, I've made some slight edits to correct LATEX and punctuation. Please double-check that this accurately reflects your intentions. – Sycorax May 24 '13 at 15:13