I am attempting to understand what I think is a misunderstanding of contingency table analysis among some biologists. In particular, I see the use of contingency tables for the analysis of numeric count data when studies lack replication. This seems inappropriate to me, but I typically use GLMs for binary and count data and want to make sure I am not missing some aspects of the use of contingency tables and chi squared tests. Below I will describe the general situation and outline a particular example.

My specific questions are

  • Is my understanding/characterization of contingency table analysis correct?
  • Is there a way the data I am describing could be analyzed correctly with a contingency table?
  • How can I refine my explanations to best convey to students and collaborators the correct use of these methods?

Contingency table analysis of "counts" I think the problem might originate because when describing contingency table analyses, the term "counts" is frequently used to refer to the elements of each cell in a table. For example, the online Handbook of Biological Statistics describes a chi square test as appropriate "when you have 1 nominal variable with 2 or more values ..." and "You compare the observed counts of observations in each category with the expected counts." That is, when you have a nominal response variable (ie red, yellow, orange flowers) and you have counts of the number of times independent observations or trials have taken on each of those values.

In my experience, I think researchers can forget the "when you have one nominal variable" part and see only the word "counts"; that is, they use a chi-square or other methods (Fisher's exact test, G-test) on count data when they should use a linear model approach (t-test, regression, etc).

MADE UP EXAMPLE Say I go into a field and select a random point. Within 10 meters of that point there are 90 flowers. I go into a nearby forest, select a random point, and there are 30 flowers within 10 meters of the point. A chi square test in R


Gives me a tiny p value of 4*10^-8. However, as I understand it, this test is inappropriate and there is no valid way to analyze these data. I have "counts" as the definition of a chi square test says I need, but they are not counts of the number of times a nominal response variable occurs; they are unbounded numeric data. To carry out a valid analysis, I would need replicated samples from each habitat. Is this correct?

REAL EXAMPLE In this paper on tropical birds, the author's used nets to capture birds at 2 different elevations. They want to know whether birds are more abundant at one elevation or the other. At each elevation they had 1 sampling location (no replication at a given elevation). At one elevation they caught 128 birds and at the other they caught 141 (Their Table 1, bottom row), and they want to test for a difference using a chi-squared test (Their Table 2, bottom row)). I believe if they had replicated sites they would've done a t-test, but since they only have one sampling location at each elevation they can't calculate a mean capture rate and have erroneously identified a chi-squared test -- perhaps because it works on "counts" -- as an alternative test.

EXPECTED COUNTS BASED ON SAMPLING EFFORT The previous example is not exactly how the authors carried out their analysis. In addition to the raw counts at each elevation(128 vs 141) they used information on the amount of sampling effort to formulate their expected counts. The amount of time the researchers spent catching birds was different between elevations; at one site they spent 287 hours and the other 157 (these are referred to as "net hours" - number of hours*number of nets). They state "comparisons were based on actual numbers (not rates) with expected values based on number of net hours." That is, they want to account for different amounts of sampling effort: 128 birds/287 net hours at one elevation vs. 141 birds/157 net hours at the other. They unfortunately to not report their test statistic nor exact p value, just p < 0.001, but I believe the R code pasted below replicates their workflow, where the expected number of birds at a site (E.i) was calculated as

E.i = (total birds caught during study)/(total hours of effort)*(hours of effort at site i)

I may very well be missing some aspect of how chi squared goodness of fit tests are used, but this does not seem to be correct. I believe the authors misunderstand the purpose of formulating "expected counts," again being mislead by the word "counts." As I understand it, the things being counted should be able to take on one discrete value or another, and the "expected count" is the number of times, given a number of trials or events, that one value should occur instead of the other. The expectation can be that a certain ratio occurs that is different from 50:50, say a sex ratio of 75:25. I believe this is where the authors think their weighting by sampling effort comes in, resulting in more captures where more effort was expended.

Calculations in R

obs1 <- 128; obs2 <- 141   
obs.tot <- obs1+obs2    

#sampling effort ("net hours")           
nethours1 <- 287; nethours2 <- 157
nethours.tot <- nethours1+nethours2

#expected counts based on effort
expect1 <- obs.tot/nethours.tot*nethours1
expect2 <- obs.tot/nethours.tot*nethours2

#ch2 test statistics: 
ch2 <- sum(c(((expect1-obs1)^2)/expect1, ((expect2-obs2)^2)/expect2))

#p value
pchisq(ch2,1,lower.tail = FALSE)
  • 2
    $\begingroup$ This is quite dense. I'll just peel off one concrete example. The chi-square test on 90, 30 as observed counts is testing a null hypothesis that the categories are equally common and under that the expected frequencies are 60, 60. If the null hypothesis provides a helpful benchmark or reference situation, then the test makes sense, but otherwise it is irrelevant and some other benchmark should be used, or some other method. The test doesn't know how far the data are, or could be, replicated. I've seen significance testing of Titanic survivals; no replication is possible or ethical there. $\endgroup$
    – Nick Cox
    Feb 27, 2017 at 18:33
  • $\begingroup$ Thanks for wading into my long question! You imply there is/could be validity to a chi2 test on my fake flower data. I didn't think there was at all, so I'll try to clarify things to improve my understanding: there are 2 cells: "# of flowers counted in field" & "# counted in forest." Each flower in not an independent event that could occur in either category, so I see no application for a chi2 test (but I am a biologist). In contrast, say I had 120 butterflies & released them. 90 settled in the field, 30 in the forest. Then I have independent events & a chi2 test would be valid. $\endgroup$
    – N Brouwer
    Feb 27, 2017 at 19:05
  • 1
    $\begingroup$ You seem to be making my point in a different way. The test is there for situations in which it makes sense, an empty but nevertheless also essential point. The null hypothesis is not a complete specification of the problem. For example, if I count two-legged people and statistically-minded people, the two categories are not disjoint and the test then makes no sense. But I don't see any statistical difference at all between your flowers and your butterflies, assuming it's different flowers and different butterflies in each count. $\endgroup$
    – Nick Cox
    Feb 27, 2017 at 19:20
  • $\begingroup$ Thanks for the further clarification. You state "If the null hypothesis provides a helpful benchmark or reference situation..."; I think that statement helps me identify my focal concern: biologists (including me) sometimes don't understand the appropriate null for a chi2 test. My flower & butterfly examples have 2 different nulls. For the flowers, Ho = "flowers are equally common in fields and forests," w/ a numeric response variable. For the butterflies the Ho I intended was: "if given a choice, butterflies are equally attracted to fields & forests," a binary response. $\endgroup$
    – N Brouwer
    Feb 27, 2017 at 19:47
  • $\begingroup$ Your second null hypothesis may well be the biological motivation but it still has the same statistical translation as the first null hypothesis. $\endgroup$
    – Nick Cox
    Feb 28, 2017 at 0:04

1 Answer 1


It may help to consider that there can be different kinds of counts. For example, if I flip a coin $n$ times, I can count how many heads I get. This could be $0$, $n$, or any whole number in between. Such data are binomial (or possibly multinomial). The point is that they have an intrinsic upper bound.

It is also possible for a count not to have a meaningful upper bound, such as the number of times a bird flies by my window over my lunch hour. There can be any number of birds, and some might fly back and forth many times. So while common sense suggests the number isn't really likely to be $18,\!000,\!000$, it could be an arbitrarily large number in the sense that I really cannot get $n+1$ heads out of $n$ flips.

This distinction is important for thinking about whether the chi-squared test makes sense. You shouldn't be using the chi-squared test if the upper bound isn't clear and known. Consider how the chi-squared test works when applied to a contingency table: glossing over quite a bit of detail, it divides various counts by the sum of the counts in the table to compute proportions. The test assumes the data are binomial / multinomial and that the table sum was the largest possible value. Mathematically, it is trivially true that the counts in the table will not be greater than the table sum, but those assumptions are nonetheless plainly incorrect when the upper limit is not bounded. When there is no upper bound, as in your examples, you aren't working with proportions of a total, but rates of some kind (e.g., flowers per $m^2$, etc.). At any rate, I agree with you that the chi-squared test is inappropriate for both your made-up flower example, and your real-world bird example.

On the other hand, it isn't quite true that such data cannot be analyzed. The most basic distribution for counts that are not bounded from above is the Poisson distribution. This often serves as a default distribution for such counts. A feature of the Poisson distribution is that the variance is equal to the mean. Thus, if you are willing to assume that the data would be distributed as Poisson in the population, you don't need $\ge 2$ counts in each condition, you could have only one and can still conduct a test (cf., my answer here: Two rate chi squared tests infection rate).

#   Comparison of Poisson rates
# data:  c(90, 30) time base: 1
# count1 = 90, expected count1 = 60, p-value = 3.773e-08
# alternative hypothesis: true rate ratio is not equal to 1
# 95 percent confidence interval:
#  1.964990 4.699155
# sample estimates:
# rate ratio 
#          3 

You can further add an offset to account for sampling effort. (An implication here is that you should not use a linear model or t-test.) However, this is flatly implausible. Count data are almost always overdispersed in practice, which means the p-value from this test is too low. So one way to use such a test is to put a plausible lower bound on the p-value.

  • 3
    $\begingroup$ Could you explain why "You shouldn't be using it if the upper bound isn't clear and known"? What is the reason for this? It seems you might have a particular context or interpretation in mind; describing it would help us understand the scope of your opinion. $\endgroup$
    – whuber
    Feb 27, 2017 at 19:59
  • 1
    $\begingroup$ @whuber, by "it", I meant the chi-squared test for contingency tables. I added some more information to flesh out the idea. See if you think it is clear enough for people now. $\endgroup$ Feb 27, 2017 at 20:15
  • 1
    $\begingroup$ I think the untransformed thing is just a convenience feature of the function, @NBrouwer. The idea of not using this test when overdispersion is common is very reasonable to me. $\endgroup$ Feb 27, 2017 at 20:51
  • 2
    $\begingroup$ @NBrouwer, the negative binomial has an independent parameter for the dispersion. If we take the count as our best (more realistically, only possible) guess for the mean count in the population, we still have no information about the dispersion. So you cannot use the NB dist w/ only 1 count per group. $\endgroup$ Feb 27, 2017 at 21:27
  • 2
    $\begingroup$ I don't really follow you, @whuber. Certainly chi-squared tests show up in other contexts (eg, differences in dispersion b/t nested models), but the chi-squared test for contingency tables assumes the data are multinomial. If they aren't, it's inappropriate. Consider the way the test was applied: it assumes there were a certain number of possible flowers that could have ended up in only 1 of 2 locations. That isn't what's going on. Even if there is a biological constraint at the 2 locations st flowers crowd out other plants or vice versa, the total # of flowers isn't the right total to use. $\endgroup$ Feb 27, 2017 at 21:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.