Is 2AFC a good way to determine the preferred choice? We have two stimuli, A and B, for which we would like to determine if there is a significant preference among users. (For some background info: A and B are 3D scenes rendered with different stereoscopic projection methods. Participants sit in front of the screen at the correct viewpoint. They will not know exactly what the experiment is about, and they will not see which stimulus is which.)
We are computer scientists, so please forgive any naive questions.
We were thinking of kind of a (temporal) 2AFC paradigm. The idea would be to present users stimulus A or B on the screen, and let users switch back and forth as often and as long as they want until they can tell which one they prefer.
Our hypothesis is that users will prefer stimulus B, i.e., one of the two kinds of stereoscopic projections.
So, my question is: is that a good and valid way to survey users' preference between the two ways of stereoscopic rendering?
 A: Suppose you have $n = 25$ subjects and $x = 18$ of them prefer B.
Then you can use an exact binomial test of the hypothesis $H_0: p_B = p_A$ that the two methods are equally preferred against the alternative $H_a: p_B > p_A.$
In R statistical software an exact binomial test would look like this:
binom.test(18, 25, alt="g")

        Exact binomial test

data:  18 and 25
number of successes = 18, number of trials = 25, p-value =  $0.02164
alternative hypothesis: 
 true probability of success is greater than 0.5
95 percent confidence interval:
 0.5377911 1.0000000
sample estimates:
probability of success 
                  0.72 

The P-value is the probability of getting 18 or more in favor of B if
the two methods are equally preferred. Because the P-value $0.02164 < 0.05 = 5\%$ we say that the sample proportion $p_B = 18/25 = 0.72$ is significantly larger than $1/2$ at the 5% level of significance.
If the random variable $X$ is the number of votes for B out of $n = 25,$
then the null hypothesis says that $X$ has the distribution $\mathsf{Binom}(n=25, p = 1/2).$ Then we can say that
$$P(X \ge 18\,|\,p_B=1/2) = 1 - P(X \le 17\, | \,p_b = 1/2) = 0.0216,$$
where the probability is found using R, in which pbinom designates the cumulative
distribution function (CDF) of a designated binomial distribution. This is
how the P-value is computed.
1 - pbinom(17, 25, 1/2)
[1] 0.02164263

The output from binom.test also gives a 95% confidence interval $(0.538, 1.000).$ for $p_B,$ another indication that $p_B > 1/2.$
Some textbooks use a normal distribution with mean $\mu = np = 25/2 = 12.5$ and standard deviation $\sigma = \sqrt{np(1-p)} = \sqrt{25/4} = 2.5$ to approximate the binomial distribution $\mathsf{Binom}(n=25, p_B=1/2).$
Especially for large $n$ and $p$ near $1/2,$ such approximations give useful results. Below the normal approximation of the P-value is $0.02275,$ which would also lead to rejection of $H_0$ in favor of the
alternative that $\hat p_B$ is significantly larger than $1/2.$
1 - pnorm(17.5, 12.5, 2.5)
[1] 0.02275013

You can find additional presentations of this test in elementary statistics texts and in various pages online.
