36

We use these tests for different reasons and under different circumstances. $z$-test. A $z$-test assumes that our observations are independently drawn from a Normal distribution with unknown mean and known variance. A $z$-test is used primarily when we have quantitative data. (i.e. weights of rodents, ages of individuals, systolic blood pressure, etc.) ...


17

Some would argue that even if the second margin is not fixed by design, it carries little information about the lady's ability to discriminate (i.e. it's approximately ancillary) & should be conditioned on. The exact unconditional test (first proposed by Barnard) is more complicated because you have to calculate the maximal p-value over all possible ...


17

There are some common misunderstandings here. The chi-squared test is perfectly fine to use with tables that are larger than $2\!\times\! 2$. In order for the actual distribution of the chi-squared test statistic to approximate the chi-squared distribution, the traditional recommendation is that all cells have expected values $\ge 5$. Two things must be ...


17

There may be some confusion about term "Barnard"s test or "Boschloo"s test. Barnard's exact test is an unconditional test in the sense that it does not condition on both margins. Therefore, both the second and third bullets are Barnard's test. We should instead write: Both margins fixed (Hypergeometric Dist'n)→Fisher's exact test One margin fixed (Double ...


15

In my opinion, the source that you link to is wrong in that it is confusing conditioning with assumptions. Fisher's exact test conditions on the margin totals, meaning that it does not use any information about independence that might be inferable from the margin totals. In probably theory, once you condition on a random variable, the random variable is then ...


12

Your 2-sided test implicitly allots exactly half of your 5% significance level to "masks are harmful" ($M_-$) and the other half to "masks are beneficial" ($M_+$). To a Bayesian like Taleb that might suggest that you aren't properly thinking about your prior, because it implies that the amount of evidence it would take you to accept $M_-$ ...


11

Fisher's exact test works by conditioning upon the table margins (in this case, 5 males and females and 5 soda drinkers and non-drinkers). Under the assumptions of the null hypothesis, the cell probabilities for observing a male soda drinker, male non-soda drinker, female soda drinker, or female non-soda drinker are all equally likely (0.25) because of the ...


11

The first test tells you that the odds ratio between A and B, ignoring C, is different from 1. Looking at the stratified analysis helps you decide whether it's all right to ignore C. The CMH test tells you that the odds ratio between A and B, adjusting for C, is different from one. It returns a weighted average of the stratum-specific odds ratios, so if ...


11

From the help page for fisher.test(): Note that the conditional Maximum Likelihood Estimate (MLE) rather than the unconditional MLE (the sample odds ratio) is used.


11

In the $2\times 2$ case the distributional assumption is given by two independent binomial random variables $X_1 \sim Bin(n_1, \theta_1)$ and $X_2 \sim Bin(n_2, \theta_2)$. The null hypothesis is the equality $\theta_1=\theta_2$. But Fisher's exact test is a conditional test: it relies on the conditional distribution of $X_1$ given $X_1+X_2$. This ...


11

This is a good idea, but the lady knows that there are 4 cup of tea for each type. This is a valuable information for the lady, which makes things wrong if we model the process via a binomial distribution. The problem is that the variables (successes at each trial) you want to consider are not independent and identically distributed. I think you have ...


10

You have a few issues here. First, understanding what each test is doing, and second interpreting the p-values. First, each test has different underlying assumptions. The likelihood ratio test statistic is formed by taking the log of the ratio of the likelihood under the null model, divided by the alternative model. The test statistic is approximately ...


10

Fisher's so-called "exact" test makes the same kind of subtle assumptions that $\chi^2$ tests make. The two variables being assessed for association are truly polytomous all-or-nothing variables such as dead/alive US/Europe. If one or both of the variables is a simplification of an underlying continuum, categorical data analysis should not be undertaken at ...


10

The solution depends intimately on how the data were collected and summarized. This answer takes you through a process of thinking about the data, analyzing them, reflecting on the results, and improving the test until some insight is achieved. Along the way we develop and compare five variants of the $\chi^2$ test. Fisher's test is not applicable because ...


10

The problem is the data are discrete so histograms can be deceiving. I coded a simulation with qqplots that show an approximate uniform distribution. library(lattice) set.seed(5545) TotalNo=300 TotalYes=450 pvalueChi=rep(NA,10000) pvalueFish=rep(NA,10000) for(i in 1:10000){ MaleAndNo=rbinom(1,TotalNo,.3) FemaleAndNo=TotalNo-MaleAndNo MaleAndYes=...


10

Having observed zeros is not an issue for a Fisher Exact test -- nor indeed is it a problem for a chi-squared test (it's not clear why you think this would be a difficulty with an exact test; if you can clarify the source of your concern, additional explanation/clarification may be possible). An entire row or column of zeros might potentially be an issue for ...


9

It's hard to read this quotation & not surmise that the author considers it a mere blunder to use Fisher's Exact Test when the marginal totals of a contingency table are not fixed by design. "Fisher's original use" of the test must refer to the famous lady tasting tea who "has been told in advance of what the test will consist, namely that she will be ...


8

What you are asking for here is a post-hoc power analysis. (More specifically, "the probability of correctly rejecting the null hypothesis" is the power, and 1-power is beta, "the probability of a type-II error". You ask for both, but we only need one to know the other.) We take your existing dataset as the alternative hypothesis / model of the true data ...


8

You need McNemar's test (http://en.wikipedia.org/wiki/McNemar%27s_test , http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3346204/). Following is an example: 1300 pts and 1300 matched controls are studied. The smoking status is tabled as follows: Normal |no |yes| Cancer|No |1000|40 | |Yes |200 |60 | Each entry of the table ...


8

The fisher's exact test in R by default tests whether the odds ratio associated with the first cell being 1 or not. That said, you can interpret the odds ratio 0.53 as: the odds of being male for a non-overwieght subject is 0.53 times that for an overweighted subject. Note the p-value is significant and the confidence interval doesn't contain 1. Therefore, ...


8

You have chosen to do a one-sided test and, obviously, order is important in a one-sided test. Your first call to fisher.test is testing the null hypothesis Pct1 = Pct2 vs the alternative that Pct1 < Pct2. The second call is testing the same null vs the alternative that Pct2 < Pct1. The two alternatives are opposites of one another, so they give p-...


8

This is definitely an ongoing debate in the literature, but at this point the evidence points to using paired analysis to compute standard errors and p-values. Although the goal of matching is to arrive at two samples that mimic a randomized control trial, not a paired-randomized control trial, matching does still induce a covariance between the outcomes ...


7

I know I am several months late, but just want to respond to the other answers. All answers use simulations and/or claim the exact Fisher calculation is too computationally intensive. If you code this efficiently, you can get an exact computation very quickly. Below is a comparison time of the sample code fisherpower() function vs. the power.exact.test() ...


6

As already pointed out in my comment referring to the original question, your preferred null hypothesis "color distribution in Urn 1 is equal to color distribution in all Urns combined" is equivalent to the null hypothesis "color distribution in Urn 1 is equal to color distribution in Urn 2-7". The former recycles observations in Urn 1, destroying ...


6

A chi-squared test will be simplest and most appropriate. Fisher's exact test tests for differences conditional on fixed margins, which is almost certainly inappropriate here. Logistic regression would be fine, but chi-squared would be simpler; also, LR is really assessing smoking as a function of your groups, which does not quite conceptually match your ...


6

(From a quick skim of the article, I'm not a fan of the analyses.) Fisher's exact test is often ill used, in my opinion (cf., here: Given the power of computers these days, is there ever a reason to do a chi-squared test rather than Fisher's exact test?). Fisher's test is based on the assumption that that the marginal counts were fixed in advance and the ...


6

In any randomization test, the probability is the proportion of possible outcomes (given the data but not given the assignment to conditions) as extreme or more extreme than the actual data. If the one in the data is the least extreme, p = 1. It is more of a proportion than a probability in the mathematical sense. The ratios of dataset 1 to 2 are 47:1 and ...


6

The Fisher's Exact Test is a test of the odds ratio. If the contingency table has values enumerated by: $$\begin{array}{c|cc} & Y & \bar{Y} \\ \hline X & a & b \\ \bar{X} & c & d \end{array}$$ Then the odds ratio is given by $$OR = \dfrac{ad}{bc}$$ And you can see the OR is the same if the contingency table is "transposed", $OR = ...


6

A glib answer is that they probably just plugged their numbers into a power calculator. I've attached a screenshot re-creating this power analysis in G*Power 3.1, a freely available power calculator. Note to match their result of 621 I had to go to "Options" and select "Maximize Alpha". The paper says "We anticipated that illness compatible with Covid-19 ...


5

It sounds like you are asking a lot of different questions here. My question is: how should I interpret the p value? I don't understand what is that referred to. The null hypothesis for Fisher's Exact test is that the groups do not affect the outcome, i.e. that they are independent. Rejection of the null hypothesis indicates the outcome (a, b, or c) is ...


Only top voted, non community-wiki answers of a minimum length are eligible