I am attempting to understand what I think is a misunderstanding of contingency table analysis among some biologists. In particular, I see the use of contingency tables for the analysis of numeric count data when studies lack replication. This seems inappropriate to me, but I typically use GLMs for binary and count data and want to make sure I am not missing some aspects of the use of contingency tables and chi squared tests. Below I will describe the general situation and outline a particular example.
My specific questions are
- Is my understanding/characterization of contingency table analysis correct?
- Is there a way the data I am describing could be analyzed correctly with a contingency table?
- How can I refine my explanations to best convey to students and collaborators the correct use of these methods?
Contingency table analysis of "counts" I think the problem might originate because when describing contingency table analyses, the term "counts" is frequently used to refer to the elements of each cell in a table. For example, the online Handbook of Biological Statistics describes a chi square test as appropriate "when you have 1 nominal variable with 2 or more values ..." and "You compare the observed counts of observations in each category with the expected counts." That is, when you have a nominal response variable (ie red, yellow, orange flowers) and you have counts of the number of times independent observations or trials have taken on each of those values.
In my experience, I think researchers can forget the "when you have one nominal variable" part and see only the word "counts"; that is, they use a chi-square or other methods (Fisher's exact test, G-test) on count data when they should use a linear model approach (t-test, regression, etc).
MADE UP EXAMPLE Say I go into a field and select a random point. Within 10 meters of that point there are 90 flowers. I go into a nearby forest, select a random point, and there are 30 flowers within 10 meters of the point. A chi square test in R
Gives me a tiny p value of 4*10^-8. However, as I understand it, this test is inappropriate and there is no valid way to analyze these data. I have "counts" as the definition of a chi square test says I need, but they are not counts of the number of times a nominal response variable occurs; they are unbounded numeric data. To carry out a valid analysis, I would need replicated samples from each habitat. Is this correct?
REAL EXAMPLE In this paper on tropical birds, the author's used nets to capture birds at 2 different elevations. They want to know whether birds are more abundant at one elevation or the other. At each elevation they had 1 sampling location (no replication at a given elevation). At one elevation they caught 128 birds and at the other they caught 141 (Their Table 1, bottom row), and they want to test for a difference using a chi-squared test (Their Table 2, bottom row)). I believe if they had replicated sites they would've done a t-test, but since they only have one sampling location at each elevation they can't calculate a mean capture rate and have erroneously identified a chi-squared test -- perhaps because it works on "counts" -- as an alternative test.
EXPECTED COUNTS BASED ON SAMPLING EFFORT The previous example is not exactly how the authors carried out their analysis. In addition to the raw counts at each elevation(128 vs 141) they used information on the amount of sampling effort to formulate their expected counts. The amount of time the researchers spent catching birds was different between elevations; at one site they spent 287 hours and the other 157 (these are referred to as "net hours" - number of hours*number of nets). They state "comparisons were based on actual numbers (not rates) with expected values based on number of net hours." That is, they want to account for different amounts of sampling effort: 128 birds/287 net hours at one elevation vs. 141 birds/157 net hours at the other. They unfortunately to not report their test statistic nor exact p value, just p < 0.001, but I believe the R code pasted below replicates their workflow, where the expected number of birds at a site (E.i) was calculated as
E.i = (total birds caught during study)/(total hours of effort)*(hours of effort at site i)
I may very well be missing some aspect of how chi squared goodness of fit tests are used, but this does not seem to be correct. I believe the authors misunderstand the purpose of formulating "expected counts," again being mislead by the word "counts." As I understand it, the things being counted should be able to take on one discrete value or another, and the "expected count" is the number of times, given a number of trials or events, that one value should occur instead of the other. The expectation can be that a certain ratio occurs that is different from 50:50, say a sex ratio of 75:25. I believe this is where the authors think their weighting by sampling effort comes in, resulting in more captures where more effort was expended.
Calculations in R
obs1 <- 128; obs2 <- 141
obs.tot <- obs1+obs2
#sampling effort ("net hours")
nethours1 <- 287; nethours2 <- 157
nethours.tot <- nethours1+nethours2
#expected counts based on effort
expect1 <- obs.tot/nethours.tot*nethours1
expect2 <- obs.tot/nethours.tot*nethours2
#ch2 test statistics:
ch2 <- sum(c(((expect1-obs1)^2)/expect1, ((expect2-obs2)^2)/expect2))
pchisq(ch2,1,lower.tail = FALSE)