# Guessing test question answers from scores

My teacher likes to give online quizzes that are about 20-30 questions long. Every student has the same questions in the same order. We are not told after taking the quiz which questions we got wrong, but the system does tell us our score.

That made me curious as to inferring the answers statistically. (Note that this is not cheating, as the teacher encourages heavy collaboration.) My idea is this: when a courageous person is confident of his answers, he submits the quiz. Each answer for each question has a fitness score. Each time someone gets the result after taking the quiz, the square of this quiz score is added to the fitness score of every answer he selected. After some tries, we will have a good prediction of the right answers for all of the questions.

Is this statistically rigorous? If not, are there better procedures?

• Where did you come up with the squaring idea? That is, on what basis do you suppose that the squares--as opposed, say, to their exponentials or logarithms or whatever--ought to work well for this purpose? – whuber Feb 20 '14 at 21:17
• @whuber I have absolutely no idea. It's just intuition. Obviously, as I have no experience whatsoever in statistics, I wanted to get some expert advice. – Simon Kuang Feb 20 '14 at 21:24
• @Glen_b I don't know how to do that. Could you explain in an answer? – Simon Kuang Feb 20 '14 at 22:29
• @glen_b I have worked out a nice strategy for a fairly general situation (multiple choice or true/false answers, with or without points deducted for guessing). Although probabilities of correctness are involved, Bayes' Theorem is not. (The flavor is closer to that of using error-correcting codes to recreate digital signals that have been degraded during transmission.) However, I do not care to participate in what amounts to gaming the assessment system and am content to leave everyone believing that an algorithm like the one originally proposed would be effective :-). – whuber Feb 21 '14 at 22:22
• It's a very interesting problem, but ... now that I've figured out more about how I'd do this at least reasonably well under different sets of assumptions about how the co-operation operates (were I in the same situation and inclined to game the system), I'm not sure I want to explain in too much detail exactly to do. Bayesian stats does still enter into part of the approaches I came up with, but it's not especially critical. – Glen_b Feb 22 '14 at 3:29