# Performing chi-squared test of significance with zero (0) observed frequency in the contingency table

I am trying to find if the flag is significantly affecting the groups distribution. I am trying to perform the chi-squared test but it is throwing a NaN value (as expected because 0 observed frequency for some groups). But then how to find if the 2 flags are really having 2 different distributions. Should I just remove the groups with 0 in either flag?

Sample of the contingency table (contingency_table_wide)

             groups
flag    0-1    1-4    5-9  10-14  15-19  20-24  25-29  30-34
0      2      1      0  28798 218272 464149 519604 412537
1      0      0      0   4552  66845 157689 147428  99612


Code

contingency_table <- xtabs(~flag+groups, data=Result_table)
# xtabs is from stats package
contingency_table_wide <- as.data.frame.matrix(xtabs(
Freq ~ flag + groups, data=contingency_table))
# for visual purposes
summary(contingency_table)


Output

Number of cases in table: 3173422
Number of factors: 2
Test for independence of all factors:
Chisq = NaN, df = 18, p-value = NA
Chi-squared approximation may be incorrect

• Having some observed 0's isn't necessarily a problem in any way at all. The problem you're having occurs when you get an entire row or column of 0's. See here or here or here Nov 24, 2016 at 9:45
• @Glen_b I just realized that seeing one of your other answers. I used drop.unused.levels = TRUE in xtabs() and it removed the column (5-9) with all 0s. Will that give a valid chi-square statistic? Also, I am getting p-value as 0 after removal so that's no indication for something gone wrong?. Nov 24, 2016 at 9:49
• 1. Which answer did you see? 2. There's almost no information in the three low column categories because the column totals are so small. Note that combining the column of zeros with any other column is identical to dropping it, Unless it makes no sense to do so for your problem, I'd suggest combining the three leftmost columns. 3. With such huge counts you should expect extremely tiny p-values -- even very small effects will be detectable at huge sample sizes. Nov 24, 2016 at 9:58
• I'm not sure what you're asking there. Combining adjacent columns purely on the basic of the column totals (i.e. without reference to the pattern of values) should present no problem. If you're asking about whether - even after combining - the test statistic should be approximately distributed as chi-square in spite of two low expected values under the null (one a bit less than 1), that should be okay. Nov 24, 2016 at 10:14
• The issue of large sample sizes and hypothesis testing has been addressed many, many times on site. It's not unique to chi-square. [Making your hypothesis test inconsistent by throwing out most of the data any time the sample size gets large seems like a really bad idea -- that you would consider it suggests that you shouldn't be using hypothesis tests at all. What are you really trying to find out?] Nov 24, 2016 at 11:25

chisq.test(collapsed_table[, 2:3], sim=TRUE)

Pearson's Chi-squared test with simulated p-value (based on 2000
replicates)

data:  collapsed_table[, 2:3]
X-squared = 7358.8, df = NA, p-value = 0.0004998


But I will look into another idea. Your column names indicates that you have grouped (binned) a continuous variable, it might have been better to avoid that if possible. So I introduce a variable x by using the interval midpoints as values, and the use the R package mgcvto fit a binomial (logistic) smooth regression. Smoothing then is a way to solve the problem of low counts. This gives

mod0 <- mgcv::gam( cbind(flag0, flag1) ~ s(x, k=3),
data=as.data.frame(mydata), family=binomial)
summary(mod0)

Family: binomial

Formula:
cbind(flag0, flag1) ~ s(x, k = 3)

Parametric coefficients:
Estimate Std. Error z value Pr(>|z|)
(Intercept)  1.90805    0.01306   146.1   <2e-16 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Approximate significance of smooth terms:
edf Ref.df Chi.sq p-value
s(x)   2      2   6718  <2e-16 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

R-sq.(adj) =  0.886   Deviance explained = 91.4%
UBRE = 80.934  Scale est. = 1         n = 8


and the following plot:

In this case a simple generalized linear model (R function glm with a quadratic term for x would have given a very similar result.

R code to read the data and make tables used above:

data_text <- "NA       0.5     2.5    7   12     17     22     27     32
0      2      1      0  28798 218272 464149 519604 412537
1      0      0      0   4552  66845 157689 147428  99612 "

mydata <- matrix(scan(textConnection(data_text), sep=""), nrow=3, byrow=TRUE)
mydata <- t(mydata)
mydata <- mydata[-1, ]
colnames(mydata) <- c("x", "flag0", "flag1")
dimnames(mydata)[[1]] <- as.character(mydata[, 1])

collapsed_table <- mydata[-(1:3), ]
collapsed_table <- rbind(mydata[1, ] + mydata[2, ] + mydata[3, ],
collapsed_table)

• Much simpler: X <- matrix(c(0 , 2 , 1 , 0 , 28798, 218272, 464149, 519604, 412537, 1 , 0 , 0 , 0 , 4552 , 66845, 157689, 147428 , 99612 ), 9); Y <- X[rowSums(X) > 0, ]; chisq.test(Y, simulate.p.value = TRUE)
– whuber
Aug 7, 2023 at 13:27