Barnard's test is used when the nuisance parameter is unknown under the null hypothesis.
However in the lady tasting test you could argue that the nuisance parameter can be set at 0.5 under the null hypothesis (the uninformed lady has 50% probability to correctly guess a cup).
Then the number of correct guesses, under the null hypothesis, becomes a binomial distribution: guessing 8 cups with 50% probability for each cup.
In other occasions you may not have this trivial 50% probability for the null hypothesis. And without fixed margins you may not know what that probability should be. In that case you need Barnard's test.
Even if you would do Barnard's test on the lady tasting tea test, it would become 50% anyway (if the outcome is all correct guesses) since the nuisance parameter with the highest p-value is 0.5 and would result in the trivial binomial test (it is actually the combination of two binomial tests one for the four milk first cups and one for the four tea first cups).
> library(Barnard)
> barnard.test(4,0,0,4)
Barnard's Unconditional Test
Treatment I Treatment II
Outcome I 4 0
Outcome II 0 4
Null hypothesis: Treatments have no effect on the outcomes
Score statistic = -2.82843
Nuisance parameter = 0.5 (One sided), 0.5 (Two sided)
P-value = 0.00390625 (One sided), 0.0078125 (Two sided)
> dbinom(8,8,0.5)
[1] 0.00390625
> dbinom(4,4,0.5)^2
[1] 0.00390625
Below is how it would go for a more complicated outcome (if not all guesses are correct e.g. 2 versus 4), then the counting of what is and what is not extreme becomes a bit more difficult
(Note as well that Barnard's test uses, in the case of a 4-2 result a nuisance parameter p=0.686 which you could argue is not correct, the p-value for 50% probability of answering 'tea first' would be 0.08203125. This becomes even smaller when you consider a different region, instead the one based on Wald's statistic, although defining the region is not so easy)
out <- rep(0,1000)
for (k in 1:1000) {
p <- k/1000
ps <- matrix(rep(0,25),5) # probability for outcome i,j
ts <- matrix(rep(0,25),5) # distance of outcome i,j (using wald statistic)
for (i in 0:4) {
for (j in 0:4) {
ps[i+1,j+1] <- dbinom(i,4,p)*dbinom(j,4,p)
pt <- (i+j)/8
p1 <- i/4
p2 <- j/4
ts[i+1,j+1] <- (p2-p1)/sqrt(pt*(1-pt)*(0.25+0.25))
}
}
cases <- ts < ts[2+1,4+1]
cases[1,1] = TRUE
cases[5,5] = TRUE
ps
out[k] <- 1-sum(ps[cases])
}
> max(out)
[1] 0.08926748
> barnard.test(4,2,0,2)
Barnard's Unconditional Test
Treatment I Treatment II
Outcome I 4 2
Outcome II 0 2
Null hypothesis: Treatments have no effect on the outcomes
Score statistic = -1.63299
Nuisance parameter = 0.686 (One sided), 0.314 (Two sided)
P-value = 0.0892675 (One sided), 0.178535 (Two sided)
