# Dataset for studying and teaching Fisher's exact test

I am looking for a dataset to study and teach Fisher's exact test. In particular, I would be interested in a simple data set that contains:

1. ("truly") experimental data
2. a few additional variables that might be used to improve the estimate.

Is there anything publicly available you could think of?

• I am looking for something like the honey experiment data which seems to be a relatively simple data set of 105 observations describing the effect of honey on coughing frequencies. The original paper is here and I came across it in chapter 5 of this book) which treats Fisher's exact test.
• They also use covariates in the chapter which are supposed to raise the power of Fisher's exact test. Actually, this is what I am not really understanding, and what I would like to look into myself. So if anybody wants to/ can explain the role of covariates, or point to some further references, of in these type settings, I would be very grateful.
• Asking for datasets is off-topic here. See here opendata.stackexchange.com Nov 26, 2016 at 15:41
• Lots of biostatistics book would give you one. Nov 26, 2016 at 15:42
• This is question is based on some apparent statistical misunderstandings that we can clear up. Thus, I think it should remain open. In addition, while questions about finding particular datasets are clearly off topic here, questions about generic data that illustrate statistical ideas or tests are ambiguous w.r.t. on/off topic-ness. Nov 26, 2016 at 17:05
• @gung if it were modified to ask about those misunderstandings it would be on topic but currently it explicitly asks for a data set. Nov 27, 2016 at 1:34
• @gung I agree there's a degree of greyness; hence the discussion rather than just closing. Given your answer (which I somehow missed before) I'm inclined to leave it open. Nov 27, 2016 at 2:34

(From a quick skim of the article, I'm not a fan of the analyses.)

Fisher's exact test is often ill used, in my opinion (cf., here: Given the power of computers these days, is there ever a reason to do a chi-squared test rather than Fisher's exact test?). Fisher's test is based on the assumption that that the marginal counts were fixed in advance and the only thing that was free to vary is which cells the observations fell into, subject to the constraint that they sum to the margins. It's hard to think of many cases where that even could be true.

The one paradigmatic case where it was true was the tea tasting experiment where Fisher used it. Fisher was at a garden party where a woman was complaining that if milk was poured into a cup first, and the tea was added afterwards, it ruined the tea, but that if it was prepared correctly (i.e., the tea was poured first, and the milk second), it was fine. Others scoffed at the idea that she could tell the difference. Fisher, rising to the occasion, offered a solution: they would prepare eight cups, four each way, and the lady would state which four she thought were which. Those data could be tested with Fisher's exact test, because there four of each preparation (specified in advance of the experiment), and the lady was obliged to choose only exactly four for each category of response. It turned out that she was right every time. That is (and this is the classic dataset for Fisher's exact test):

             Preparation:
She said:   M->T    T->M   Total
M->T       4       0       4
T->M       0       4       4
Total       4       4       8

fisher.test(table)
#   Fisher's Exact Test for Count Data
#
# data:  table
# p-value = 0.02857
# alternative hypothesis: true odds ratio is not equal to 1
# 95 percent confidence interval:
#  1.339059      Inf
# sample estimates:
# odds ratio
#        Inf


There is a nice write-up of the event and Fisher's test on Wikipedia. (Let me acknowledge that I am going rather hard on Fisher's exact test here. In most cases the result—with respect to the significance—will be the same either way, and you could argue that it doesn't matter that the assumptions aren't met, but it typically has lower power than the proper alternative.)

Regarding the idea of covariates, I'm not really sure what is being referred to. Certainly Imbens and Rubin are famous statisticians, but I don't have access to the book, so I'm limited in deciphering this. At any rate, Fisher's test does not allow for covariates. If you had a binary response variable and some covariates, you would typically use logistic regression. Fisher's test is often used by people because they have very few events. If that were the case, there probably wouldn't be enough data to support the inclusion of covariatess, so the issue seems moot.

For what it's worth, you can infer the dataset from the article. In table 1, it lists the three conditions with their $n$s as: Honey (n=35); DM (n=33); Nothing (n=37). In the text above, it states that, "Fisher exact tests were used to compare adverse event rates between treatments". At the end of the results section it states:

Few adverse events occurred in this investigation. The combination of mild reactions that include hyperactivity, nervousness, and insomnia occurred in 5 patients treated with honey, 2 patients in the DM group, and no patients in the no-treatment arm (P = .04). In the honey group, the parent of 1 patient reported drowsiness and the parents of 2 patients reported stomachache, nausea, or vomiting, but these adverse events were not significant when examined separately from a statistical perspective (drowsiness, P = .65; stomachache, nausea, vomiting, P = .21).

Thus, I gather the data were:

## Patients:
Treatment:
AE:    Honey   DM   Nothing
Yes      5    2         0
No      30   31        37

## Parents (drowsiness):
Treatment:
AE:    Honey   DM   Nothing
Yes      1    0         0
No      34   33        37

## Parents (stomachache, nausea, vomiting):
Treatment:
AE:    Honey   DM   Nothing
Yes      2    0         0
No      33   33        37