I'm new to statistics so this might be a really newbie question.. I have a table in R as such:

df <- structure(list(SampleID = structure(1:6, .Label = c("A", "C", 
                "H", "K", "V", "HG"), class = "factor"), freq = c(4L, 13L, 4L, 
                8L, 4L, 1L)), row.names = c(1L, 2L, 3L, 4L, 5L, 
                6L), class = "data.frame")

I want to see if frequencies associated with SampleID's, which represent different conditions, are statistically different from each other. I thought about using the chi-squared test of independence but not sure it is the right test.

What I've done so far is convert this table into a matrix using:

df_mat <- table(df$SampleID, df$freq)



which gave me this result:

Pearson's Chi-squared test
data:  df_mat
X-squared = 18, df = 15, p-value = 0.2627

Can I say that as the p-value 0.26 is greater than the .05 significance level, we do not reject the null hypothesis that the sample type is independent of frequency?

Is there another test that would give a fair comparison of frequencies between the samples?

  • 3
    $\begingroup$ That tabulation is erroneous. You want chisq.test(df$freq) instead. Type ?chisq.test and read the "Details" section. $\endgroup$ – whuber Sep 5 '19 at 20:34

@whuber is correct. Your tabulation yields:

     1 4 8 13
  A  0 1 0  0
  C  0 0 0  1
  H  0 1 0  0
  K  0 0 1  0
  V  0 1 0  0
  HG 1 0 0  0

This treats the counts (1, 4, 8, 13) as fixed levels of a categorical variable. It seems you want to test if the counts are approximately equal. If we take the sum (26) as the total, out of which, say, 13 became c's, you can test your observed counts against a discrete uniform distribution, which is what would occur if each instance had an equal chance of becoming each sampleID. This is a one-way chi-squared test; it is also called a goodness of fit chi-squared test, as you are testing the fit of your data to the discrete uniform. In R, that would be:


#   Chi-squared test for given probabilities
# data:  df$freq
# X-squared = 15.765, df = 5, p-value = 0.007549

On the other hand, perhaps you had one subject in each condition and counted how many times that person did something. In this case, the conditions are fixed a-priori and the counts are the outcome (the opposite of before). You would need multiple participants to get a measure of variability, but if you assume the Poisson distribution (see my answer here: Help me understand poisson.test?), you could conduct fit a model, since the Poisson has a fixed dispersion. In R, this would be:

m = glm(freq~SampleID, df, family=poisson)
# Call:
# glm(formula = freq ~ SampleID, family = poisson, data = df)
# Deviance Residuals: 
# [1]  0  0  0  0  0  0
# Coefficients:
#               Estimate Std. Error z value Pr(>|z|)   
# (Intercept)  1.386e+00  5.000e-01   2.773  0.00556 **
# SampleIDC    1.179e+00  5.718e-01   2.061  0.03926 * 
# SampleIDH   -1.851e-16  7.071e-01   0.000  1.00000   
# SampleIDK    6.931e-01  6.124e-01   1.132  0.25767   
# SampleIDV   -7.448e-17  7.071e-01   0.000  1.00000   
# SampleIDHG  -1.386e+00  1.118e+00  -1.240  0.21500   
# ---
# Signif. codes:  
# 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
# (Dispersion parameter for poisson family taken to be 1)
#     Null deviance:  1.5278e+01  on 5  degrees of freedom
# Residual deviance: -8.8818e-16  on 0  degrees of freedom
# AIC: 32.151
# Number of Fisher Scoring iterations: 3

From there, you can use the null and residual deviances to conduct a test (see my answer here: Test GLM model using null and model deviances). In R, you would do:

1-pchisq(1.5278e+01, 5)
# [1] 0.009238243

Under either interpretation, your results are significant by conventional criteria.

  • $\begingroup$ I think this is exactly what I'm looking for!! My dataset is one subject for each condition so conditions are fixed. How would I interpret which condition is significantly different from the rest? I see there are two significant samples here but not sure what that means. $\endgroup$ – Beeba Sep 10 '19 at 20:25
  • $\begingroup$ @Beeba, you will need to use the Poisson GLM version (at bottom), not the chi-squared test at top. You should read through the linked threads. Without more data, you can't determine the variability in the response. Using the Poisson (because it has a fixed variance) really serves as a lower bound--it's very likely the true variability is larger than what's assumed (& thus your effect is less significant). Regarding your explicit question in the comment, that's just about understanding how categorical variables work, see eg here. $\endgroup$ – gung - Reinstate Monica Sep 10 '19 at 20:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.