I am trying to determine whether students perform better than chance on a quiz with $q$ multiple choice questions. Each question as 4 possible responses; one is correct and 3 are wrong. A correct answer is worth one point and a wrong answer is worth 0 points.
For example, if $q=3$, I would expect an average score of 0.75 if responses are no better than chance. (There is 1 way to get 3/3 questions correct, 9 ways to get 2/3 questions correct, 27 ways to get 1/3 questions, and 27 ways to get 0/3 questions correct. $3 \left( \frac{1}{64}\right) + 2 \left( \frac{9}{64}\right) + 1 \left( \frac{27}{64}\right) + 0 \left( \frac{27}{64}\right) = 0.75 $).
Let's say that I have 50 students take the test and they have an average score of 1.25. I'm wondering:
- How would I calculate a $p$ value for the null hypothesis "true average=0.75" and the alternative hypothesis "true average≠0.75"?
- How would I calculate a confidence interval around the observed average?
- Are there any limitations related to the underlying distribution of scores (e.g. problems if the scores have a bimodal distribution with most people getting either 0 or 2+)?
