I am trying to determine whether students perform better than chance on a quiz with $q$ multiple choice questions. Each question as 4 possible responses; one is correct and 3 are wrong. A correct answer is worth one point and a wrong answer is worth 0 points.

For example, if $q=3$, I would expect an average score of 0.75 if responses are no better than chance. (There is 1 way to get 3/3 questions correct, 9 ways to get 2/3 questions correct, 27 ways to get 1/3 questions, and 27 ways to get 0/3 questions correct. $3 \left( \frac{1}{64}\right) + 2 \left( \frac{9}{64}\right) + 1 \left( \frac{27}{64}\right) + 0 \left( \frac{27}{64}\right) = 0.75 $).

Let's say that I have 50 students take the test and they have an average score of 1.25. I'm wondering:

  1. How would I calculate a $p$ value for the null hypothesis "true average=0.75" and the alternative hypothesis "true average≠0.75"?
  2. How would I calculate a confidence interval around the observed average?
  3. Are there any limitations related to the underlying distribution of scores (e.g. problems if the scores have a bimodal distribution with most people getting either 0 or 2+)?

Since n = 50 you probably want to use a z-test. Check out this example: http://www.math.umt.edu/steele/STAT451/Lecture_Notes/chap20-451.pdf.

| cite | improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.