5
$\begingroup$

How can I interpret the following result? I have 4 groups with around 300 observations each: 

        Black  Red
Group A   296   14
Group B   292   16
Group C   301    7
Group D   289   23

I want to test whether the groups have different propensity for Red outcomes.

When I use a Pearson's chi-squared test, the p.value is 0.03, which seems to suggest that groups have different propensities. But if I model the difference as binomial, the estimated GLM does not detect significant differences across groups.

Improve this question
$\endgroup$
1
  • $\begingroup$ You should model (Event frequency) binomially one of the two columns. The Trial frequency is the column of row sums. Group is the factor. The results are the same as by using Likelihood ratio (not Pearson) chi-square to test your table. $\endgroup$
    – ttnphns
    Commented Mar 13, 2015 at 9:42

1 Answer 1

Reset to default
11
$\begingroup$

I don't see a big difference in the results: 

d = read.table(text="Group Black  Red
                         A   296   14
                         B   292   16
                         C   301    7
                         D   289   23", header=T)

chisq.test(d[,2:3])
#  Pearson's Chi-squared test
# 
# data:  d[, 2:3]
# X-squared = 8.893, df = 3, p-value = 0.03075
mod = glm(cbind(Black, Red)~Group, data=d, family=binomial)
summary(mod)
# ...
# Coefficients:
#             Estimate Std. Error z value Pr(>|z|)    
# (Intercept)   3.0513     0.2735  11.156   <2e-16 ***
# GroupB       -0.1471     0.3751  -0.392    0.695    
# GroupC        0.7099     0.4701   1.510    0.131    
# GroupD       -0.5204     0.3489  -1.491    0.136    
# ...
# 
#     Null deviance: 9.3651e+00  on 3  degrees of freedom
# Residual deviance: 1.1902e-13  on 0  degrees of freedom
# AIC: 25.699
1-pchisq((9.3651 - 1.1902e-13), df=(3-0))
# [1] 0.02481063

The GLM is, if anything, slightly more significant. I wonder if this is a confusion about how to interpret statistical output from a model with categorical variables. When you have a categorical variable, most software (including R, above) uses reference cell coding (see here). The first level of the variable becomes the intercept, and the other levels are compared to the intercept. Thus, the output shows that B, C, and D do not significantly differ from A, but that doesn't mean they don't differ from each other (C and D look like they will, e.g.). To test if the entire factor / categorical variable is significant, you need to fit a new model without that variable and perform a nested model test. Since you have only one variable, you can just calculate the significance of the whole model directly using the null and residual deviance (see here).

Improve this answer
$\endgroup$
2
  • $\begingroup$ The binomoal GLM (GzLM, as some prefer to abbreviate) is exactly the same as Likelihood ratio chi-square (not Pearson chi-square): 9.365, p=.025. Advanced modeling such as GzLM or log-linear analysis won't rely on Pearson chi-square, for sure. $\endgroup$
    – ttnphns
    Commented Mar 13, 2015 at 9:21
  • $\begingroup$ You can also avoid the manual LRT test calculations, and let R calculate the p-value directly, by using either drop1(mod, test="LRT") or anova(mod, test="LRT"). $\endgroup$
    – Karl Ove Hufthammer
    Commented Mar 13, 2015 at 11:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.