Can I do multiple binomial tests seperately of each other? in the experiment the subjects had to choose between answer A and answer B in 3 different settings. The question is, if there is a significant preference for answer A in any of the settings.
Can I just do a binomial test for all of the 3 settings? Or is there a way to link these 3 tests? Is there a multiple comparisons issue?
 A: A possible answer to your research question can be obtained by a mixed binary logistic regression. The model can be explained as follows.
Let $Y_{is}$ be your answer variable such that $Y_{is}=0$ if the answer is A and $Y_{is}=1$ if the answer is B for every individual $i=1,\ldots,n$ at any setting $s=1,2,3$. Assuming $Y_{is}$ is independent from $Y_{js}$ for all $i\neq j\in\{1,\ldots,n\}$, then we assume
\begin{align*}
Y_{is} & \sim \text{Bernoulli}(p_{is})\\
p_{is} &= \frac{\exp(\alpha_{i}+\beta_0)}{1+\exp(\alpha_{i}+\beta_0)}\\
\alpha_i&\sim N(0,\sigma_{\alpha}^2),
\end{align*}
where $\alpha_i$ are unknown random intercepts assumed to be independent.
The parameters, to be estimated from data, are $\beta_0,\sigma^2_\alpha$. In particular, $\beta_0$ is your parameter of interest: if $\beta_0\neq0$ means that the proportion of the zeros and ones (e.g. A's and B's) in your response are different. The random effects $\alpha_i$ are useful for handling the dependence in the response due to repeated sampling.
You can fit this model in R using the glmer function from the lme4 package, e.g. something like
mm <- glmer(y~1+(1|patientID), data=mydata, family=binomial)
summary(mm)

In the summary, look for the estimate of the intercept and the associate p-value or build a confidence interval. The summary will also provide an estimate for $\sigma_\alpha^2$.
