4
$\begingroup$

It is not clear to me whether we can consider the Fisher's exact test as a "parametric" or "non-parametric" one. My gut feeling is that it should be defined as "parametric" as it involves a fully specified distribution (the hypergeometric). If so, however, I would not able to find an example of a non-parametric test for 2x2 contingency tables, which makes me wonder whether the distinction can be useful at all in this case.

$\endgroup$
  • 4
    $\begingroup$ The distribution of a test statistic is unrelated to whether the setting is parametric. A setting is parametric when the probability model for the data is specified up to the determination of a finite quantity of real numbers (the "parameters"). (If you are so unfortunate as to have encountered these concepts in the context of SPSS documentation, then this definition might come as a shock. Refer to any good theoretical stats textbook or even Wikipedia.) $\endgroup$ – whuber Aug 25 '15 at 18:28
  • $\begingroup$ So maybe my question can be rephrased as it follows: Does the Fisher's exact test make any assumption about the population distribution? If yes, then it is a parametric test. If not, then it is a non-parametric test. $\endgroup$ – mrb Aug 25 '15 at 20:45
  • 4
    $\begingroup$ That's not quite the correct distinction. Most non-parametric tests make assumptions about the population distribution, too. (For instance, many assume it has a probability density function.) The distinction lies in how those assumptions are expressed. When the population is assumed to have a distribution that is a member of a finitely parameterized family, then the problem is parametric. Otherwise, it is non-parametric. $\endgroup$ – whuber Aug 26 '15 at 13:48
  • $\begingroup$ Thanks that was very useful. But then, following your reasoning, what would be your final word on the Fisher's exact test? $\endgroup$ – mrb Aug 26 '15 at 16:36
  • $\begingroup$ One point I offered as a response to anothers' answer is this: it's possible to think of Fisher's test as semi-parametric. @whuber speaks of probability models for the data. The Fisher's exact test conditions on the table margins, and has a 1-1 correspondence with the odds ratio. With that information, it's actually possible to simulate table counts according to a probability model. $\endgroup$ – AdamO May 31 '18 at 15:55
4
$\begingroup$

tl;dr: Fisher's Exact Test is nonparametric in the sense that it does not assume that the population is based on theoretical probability distributions (normal/geometric/exponential etc.), but that the data itself reflects the parameters, which is why it proceeds with the assumption that the row/col totals are fixed.

Fisher's exact test, as its name suggests, gives the exact p-value rather than an estimation based on a particular sampling distribution thought to be aligning with the variable(s).

If you have two or more variables, all categorical/nominal, and your data consists of independent observations, then you can already intuitively create a cross-tabulation to assess conditional frequencies (akin to how you would want to see overlaps in a Venn diagram). For instance, say your independent variable is gender (M/F/O) and the dependent variable is party allegiance (D/R/I).

Now let's say we do not know the probability distribution of either variable, which means that we can't just plug the data into any parametric test. (In the classical FET where it's only a 2x2 (two dichotomous variables) which you know are binomially distributed, you could proceed using the hypergeometric distribution to estimate the p-value.)

Fisher's exact test directly gives us the probability of finding a result as extreme as the one we have. In other words, it reflects how far our observed frequencies are from the expected frequencies. If gender is truly independent of party membership, then there ought to be roughly uniform distribution. (Aside: you can use the 1 sample K-S test here to test for uniform distribution.)

But Fisher's test takes all the discrete values <= the observed ones, calculates their probabilities, and adds them up to give you the p-value, which you then compare to your alpha (probability of a Type I error, i.e. mistakenly rejecting the null hypothesis of there being no association between gender and party membership).

NB that although the FET is used as a recourse to the cross-tabbed chi square test when the sample size is low, the FET has its own assumptions -- I'd use it only for MECEly organised data such that the variables are 'really' nominal in a fundamental sense and not contrived for simplicity's sake (e.g. biological sex is 'truly' nominal if we use the usual definitions, whereas 'treatment status' must never be taken to be a true nominal variable) and where the individual instances are independently recorded.

For an actual rigorous idea of what the FET entails mathematically, take a look at Weisstein's neat definition -- http://mathworld.wolfram.com/FishersExactTest.html.

$\endgroup$
  • 1
    $\begingroup$ Can you reconcile your answer with @whuber's comments above? $\endgroup$ – Glen_b Jan 21 '16 at 5:55
  • 1
    $\begingroup$ I mean that the definition of "nonparametric" @whuber discusses (which seems to be the usual one, I'd have given an very similar definition) seems to be directly at odds with both the reasoning in and the conclusion of your answer. Unless I misunderstood something in your answer, I guess. $\endgroup$ – Glen_b Jan 21 '16 at 6:08
  • 1
    $\begingroup$ "the FET does not make any assumptions about the population's probability distribution." ... it doesn't? How does it order the tables by probability without that? (In fact for the 2x2 case the tables are ordered by assuming counts in a given cell follow the hypergeometric distribution; see the wikipedia article on the Fisher exact test for example). This in turn follows from other assumptions. $\endgroup$ – Glen_b Jan 21 '16 at 6:41
  • 1
    $\begingroup$ It may be reasonable to argue for it being nonparametric, but I think it requires more than just assertion to deal with the apparent contradiction between the distribution of counts in the table being defined by a finite number of parameters for any given dimension of table (the null is certainly finite parametric and any alternative would also be finite-parametric if Fisher tests had a specific alternative, but they don't) and the implied claim (in the assertion that it's nonparametric) that the distribution of cell-counts is infinite-parametric. Some explanation would seem to be needed $\endgroup$ – Glen_b Jan 21 '16 at 6:50
  • 1
    $\begingroup$ I concede -- Fisher's original description of the test is based on the hypergeometric distribution (as I mentioned with reference to the classic 2x2 case). But given that the probabilities can be calculated "from scratch" or intuitively from the contingency table itself, I think there is a strong case to be made for it assuming no probability distribution per se. Thanks for your patience. :) $\endgroup$ – paxormatic Jan 21 '16 at 7:39
2
$\begingroup$

Fisher's exact test is a parametric test, because it does assume an underlying binomial distribution for the $2\times 2$ table. The table probabilities are then calculated conditioning on the total number of successes in an exact fashion. The term parametric refers to whether distributional assumptions are made about how the data arises, rather than, say, to whether a test statistic is calculated and then compared to some distribution (e.g. normal, t, $\chi^2$ etc.).

$\endgroup$
  • $\begingroup$ How exactly is the $2 \times 2$ table setup as a binomial distribution, as you say? The table margins are conditioned upon, aka seen as "known". The actual distribution of cell counts is taken as hypergeometric. If anything, you might argue that the distribution of cell counts is "indexed" by the odds ratio, which would make it semi-parametric. $\endgroup$ – AdamO May 31 '18 at 15:35
  • $\begingroup$ @AdamO: Before conditioning, the joint distribution is product binomial - i.e. two independent binomially distributed samples. In fact it could also be multinomial or independent Poisson counts, as you're going to condition on all marginal totals anyway. $\endgroup$ – Scortchi - Reinstate Monica May 31 '18 at 16:03
  • $\begingroup$ @Scortchi That's the case only under the null hypothesis. The parametric family in parametric tests must include cases when the alternative is true. i.e. when a hypergeometric distribution describes the outcome. Correct? However, while HG-dist'ns can be parametrized, it is not a parametric test because that is a saturated parametric model for the observable data. See my question here. $\endgroup$ – AdamO May 31 '18 at 16:07
  • $\begingroup$ @AdamO: Not sure I follow. The family here is of product-binomial distributions (with two parameters), reduced to that of Fisher's non-central hypergeometric distributions (with a single parameter) after conditioning. Inference is about the non-centrality parameter, the odds ratio, & the full apparatus of hypothesis testing, & of point & interval estimation is available - FET is just for the special case of the "nil null". How is this not a parametric approach? $\endgroup$ – Scortchi - Reinstate Monica May 31 '18 at 16:40
  • $\begingroup$ @Scortchi I'm not sure what you mean by nil-null; Can you simulate $2 \times 2$ contingency-table realizations with any cell-frequencies given the margin-total with this product-binomial model? If I can create a big enough probability model to describe any data generating process, do I call erstwhile non-parametric tests "parametric"? What if I look at the empirical distribution function as a parametric model? $\endgroup$ – AdamO May 31 '18 at 17:08
1
$\begingroup$

Consider the case of comparing two samples of dichotomous observations ("success" & "failure"), & taking one of the following approaches to defining a test statistic & its sampling distribution:

(1) Assume, under the null hypothesis, all observations are drawn independently from the same distribution & condition on the order statistic (sufficient for the distribution under the null). Decide that the count of "successes" in the first sample (or any equivalent) measures discrepancy with the null in the direction of the kind of alternatives you're interested in.

(2) Assume the observations in each sample constitute independent Bernoulli random variates. The population odds ratio is the parameter of interest & the overall odds of "success" is a nuisance parameter; the counts of "successes" in each sample are jointly sufficient & constitute binomial random variates. Condition on the total count of "successes", which is sufficient for the nuisance parameter & almost ancillary for the parameter of interest.

By now you're in the same place whether you've been shunning (1) or embracing (2) parametric assumptions while devising your test: with the count of "successes" in the first sample following a hypergeometric distribution under the null. (Putting to one side any subtle considerations that may arise from wanting to carry out a two-tailed test.) For observations that can take only two values the independently, identically distributed assumption alone entails a full parametrization of the distribution under the null after conditioning.

$\endgroup$
0
$\begingroup$

Tests that are used on nonparametric data can be used on any data normal or not normal (it is just in comparison to tests that can only be used on normal data they are less reliable). Fishers test is one of the tests that is actually prefered to use over chi-squared when the data is too small.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.