Comparing accuracy of answers to a question to expected accuracy by chance - statistically significant? I'm sure this is a very simple one, but I'm struggling to find an explanation online for something so simple.
I have a survey with 38 respondents. Each one has answered a multiple choice question with 3 possible answers, one of which is correct. I would like to know whether the success rate of my respondents can be explained by chance alone. The average accuracy of the respondents is 36.8421%. How do I work out if the gap between this and 1/3 is statistically significant? I have been looking at T-tests and chi squared, but they seem to be for more complicated data sets. Is it something really obvious?
Thank you.
 A: If there are $N$ respondents, each of whom has probability $p$ of getting the correct answer, and the answers are independent, the total number of correct answers $Y$ will have the Binomial distribution:
\begin{equation}
Y \sim \mathrm{Bin}(N,p).
\end{equation}
In particular, in this case $N=38$ and under the null hypothesis that everyone just guesses randomly (uniformly among the $3$ alternatives), 
\begin{equation}
Y \sim \mathrm{Bin}(38,1/3).
\end{equation}
Next, we compute the probability that under the null hypothesis $Y$ obtains a value as extreme or more extreme than what was observed. One-tailed test is fine in this case as there is not much reason to expect that the answerers have might have smaller-than-random chance. $N=38$ is small enough that this can be obtained by summing the possibilities (otherwise, normal approximation would be used). In MATLAB code,
1 - binocdf(13,38,1/3)

(This binocdf call computes $P(Y\leq 13)$). The result is
\begin{equation}
P(Y\geq14) \approx 0.38,
\end{equation}
so the null hypothesis cannot be rejected (with any conventionally used cutoff level). That is, we conclude that the observed data is consistent with random guessing. 
