How can we have multiple "exact" tests? Looking over the differences between Fisher's exact test and Barnard's test, it seems I'm missing something fundamental. If both are exact tests, shouldn't both give the exact and identical value for $P(data | H_0)$, in this case $P(data | p_{r1} = p_{r2})$?
 A: Not really; all an exact test needs to do is accept or reject the null hypothesis based on the actual distribution of the test statistic.  Consider the following two tests of whether the mean of a Normal distribution with known standard deviation equals zero.

*

*A z-test for whether the mean $= 0$.  No explanation needed!


*A binomial test for whether the mean (= the median) $= 0$.  Here we assume the probability of an observation being above the median $= 0.5$; this is an exact test, as the binomial distribution with probability $0.5$ is the true distribution of the number of observations above the median.
These two tests will (almost) certainly give different results when calculating the probability of the observed data under the null hypothesis, but they are both exact.
Expanding on this example to address a comment:
The key here is that, although the underlying distribution is the same in both cases, the test statistics $T_N(x)$ and $T_B(x)$ are not (regardless of whether the null hypothesis is true.)  There is no 1-1 map between the two (this is important.)  Consequently, $p(x|T_N(x))$ and $p(x|T_B(x))$ aren't guaranteed to be the same either.  The underlying distributions are the same; the distributions conditional upon the test statistic are not.
However, conditional upon the test results failing to reject the null hypothesis, in both cases we have the "estimated" mean $= 0$ and standard deviation $= 1$.  Consequently, $p(x|H_0, T_N(x)) = p(x|H_0, T_B(x))$; the test statistic is irrelevant as the distribution is fully determined (in this case) by $H_0$.  In the more general case where we have parameters $\theta$ that are not specified by the hypotheses and therefore need to be estimated, as long as a) the estimation procedure is the same regardless of the test chosen, and b) neither test rejects the null hypothesis, we have $p(x|H_0, \hat{\theta}, T_1(x)) = p(x|H_0, \hat{\theta}, T_2(x)) = p(x|H_0, \hat{\theta})$.
A: *

*The answer mostly  comes down to definition. An exact test is one that when the assumptions it makes under $H_0$ hold, will not exceed the selected type I error rate (anywhere under the null), at any given sample size [1].
For a point null (almost all the tests you're likely to be doing in pactice) it should equal the desired type I error rate if that is chosen from the available rates (with a discrete test statistic there's a finite set of  choices unless you use randomized tests).
Unsurprisingly given such a broad definition,  there's an infinite number of exact tests. For example, most common rank based tests are exact (given the earlier caveats).
While the usual chi-squared test of independence is not small-sample exact, it is possible to construct an exact test using that statistic (indeed it's an option in R's chisq.test to do that exact test).
In the same fashion, one could construct an exact test based on the G statistic or the Neyman statistic or the Freeman-Tukey statistic, or any of the other Cressie-Reed power-divergence statistics, or indeed any number of other possibilities, all for the same contingency table and under the same assumptions and the same conditioning on the margins - an infinite array of possible tests just for independence in  contingency tables.
Note that Barnard's test [2] does not condition on both margins, so it would be part of another group of tests again.


*A small clarification is needed (I presume you understood this already but the phrasing in the question may mislead some readers).
Hypothesis tests don't
evaluate $P(\text{data}|H_0)$, since significance levels are based on the probability that the test statistic will fall into the rejection region (and thence the 'as, or more extreme' phrasing for p values). Generally the test will have an explicit test statistic whose cdf can be computed exactly (at least in principle, and to any required degree of accuracy in practice) and hence explicit rejection rules obtained. From this an  exact p value will be implied. E.g. for tests where you would reject for small values of the test statistic (say T), $P(T\leq t_\text{obs})$ is the p value, and different tests will not order the possible samples the same way (e.g. an exact test based on a chi squared statistic and a Fisher exact test will not always perfectly correspond).  While a test statistic is not normally given for the Fisher exact test, it is possible to define one, even in $r\times c$ tables.
For that test, since that effective statistic is based on likelihood under the null, you could indeed consider $\text{data}|H_0$, but 'as or more extreme' includes all other tables [2] with equal or lower probability, not just that specific data table.

[1]: Further, since we don't know that the exact form of population distributions hold, typically people dont use the phrase 'exact' when there's a specific distributional form assumed (that doesn't follow from the other assumptions).
[2]: If I recall correctly, many decades later Barnard came to the conclusion that Fisher had been correct to condition on both margins in this situation.
[3]: ... with the same margins, since the test conditions on the margins. However, the test does not require one to have fixed margins  (as many texts incorrectly state).
A: What make and exact test exact is that it uses the actual distributions of the null hypothesis (e.g. binomial or hypergeometric or multinomial distributions) rather than approximations (e.g. normal or chi-square distributions) to calculate $p$-values of the probability of seeing the observed result or another result as or more extreme.
Since they often involve discrete distributions, they usually do not have a critical region with probability exactly equal to some pre-specified significance level $\alpha$, causing some further confusion over the word exact.
Different tests have different distributions underlying the null hypothesis: the Fisher test assumes a hypergeometric distribution while Barnard's test assumes two binomial distributions.  This means that they usually calculate different $p$-values.  Some tests may also have different views of what counts as more extreme.  So different tests should not be expected to give the same results, and this is a reason why you should decide which test to use before you see the data.
