What will be the correct answer, if we modify the "Best statistics question ever"?

Question

There is a popular question, called "Best statistics question ever".

If you choose an answer to this question at random, what is the chance you will be correct?

A) 25% B) 50% C) 60% D) 25%

This task is not very difficult, the correct answer is 0%. But if we modify it like this:

If you choose an answer to this question at random, what is the chance you will be correct?

A) 50% B) 25% C) 60% D) 50%

What will be the correct answer? Do we have two correct answers: 25% and 50%, or there is no correct answer, as with this two correct answers the chance of choosing the correct answer is in fact 75% (but we do not have 75% written on the desk)?

By the way. Does the answer 0% remains correct answer, the third correct answer in this case?

Answer 1

I think there is an issue of semantics here in addition to probability. Choosing at random is clear. Each of A, B, C, and D will be selected 25%. But what does it mean to be correct when you pick at random? It seems that it should mean given that you pick A does As answer give the correct % of samples that will be correct and the same for B, C, and D. So you have to count 1/4 for each correct answer and sum over all correct answers to get the correct percentage. But this leads to a circular argument. Hence the paradox. This really seems to be a question in logic rather than probability or statistics.

Answer 2

The apparent paradoxes (of logic or probability) can be resolved by framing the questions clearly and carefully.

The following analysis is motivated by the idea of defending an answer: when a test-taker can exhibit a possible state of affairs (consistent with all available information) in which their answer indeed is correct, then it should be marked as correct. Equivalently, an answer is incorrect when no such defense exists; it is considered correct otherwise. This models the usual interactions between (benevolent, rational) graders and (rational) test-takers :-). The apparent paradox is resolved by exhibiting multiple such defenses for the second question, only one of which could apply in any instance.

---

I will take the meaning of "random" in these questions in a conventional sense: to model a random choice of answer, I will write each answer on a slip of paper ("ticket") and put it in a box: that will be four tickets total. Drawing a ticket out of the box (after carefully and blindly shuffling the box's contents) is a physical model for a "random" choice. It motivates and justifies a corresponding probability model.

Now, what does it mean to "be correct"? In my ignorance, I will explore all possibilities. In any case, I take it as definite that zero, one, or even more of the tickets may be "correct." (How might I know? I simply consult the grading sheet!) I will mark the "correct" answers as such by writing the value $1$ on each correct ticket and writing $0$ on the others. That's routine and should not be controversial.

An obvious but important thing to notice is that the rule for writing $0$ or $1$ must be based solely on the answer written on each ticket: mathematically, it is a mapping (or reassignment) sending the set of listed answers ($\{.25, .50, .60\}$ in both questions) into the set $\{0,1\}$. This rule is needed for self-consistency.

Let's turn to the probabilistic element of the question: by definition, the chance of being correct, under a random drawing of tickets, is the expectation of the values with which they have been marked. The expectation is computed by summing the values on the tickets and dividing that by their total number. It will therefore be either $0$, $.25$, $.50$, $.75$, or $1$.

A marking will make sense provided that only the tickets whose answers equal the expectation are marked with $1$s. This also is a self-consistency requirement. I claim that this is the crux of the matter: to find and interpret the markings that make sense. If there are none, then the question itself can be branded as being meaningless. It there is a unique marking, then there will be no controversy. Only if two or more markings make sense will there be any potential difficulty.

Which markings make sense?

We don't even need to make an exhaustive search. In the first question, the expectations listed on the tickets are 25%, 50% and 60%. The latter is impossible with four tickets. The first would require exactly one ticket to be marked; the second, two tickets. That gives at most $3+3=6$ possible markings to explore. The only marking that makes sense puts $0$s on each ticket. For this marking, the expectation is $(0+0+0+0)/4 = 0$. That justifies the stated answer to the first question. (Arguably, the sole correct response to the first question is not to select any answer!)

In the second question, the same answers appear and once again there are six markings to explore. This time, three markings are self-consistent. I tabulate them:

Solution 1                Solution 2                Solution 3
Ticket Answer Mark        Ticket Answer Mark        Ticket Answer Mark
     A    50%    1             A    50%    0             A    50%    0
     B    25%    0             B    25%    1             B    25%    0
     C    60%    0             C    60%    0             C    60%    0
     D    50%    1             D    50%    0             D    50%    0

Therefore, there are three distinct possible definitions of "correct" in the second problem, leading to either A or D being correct (in solution 1) or only B being correct (in solution 2), or none of the answers being correct (in solution 3).

One way to interpret this state of affairs is that for each of the answers A, B, and D, there exists at least one way of marking the tickets that makes those answers correct. This does not imply that all three are simultaneously correct: they couldn't be, because $.25 \ne .50$. If you were the grader of the test, then if you marked any of A, B, or D correct, then you would not get an argument from the test-taker; but if you marked any of them incorrect, the test-taker would have a legitimate basis to dispute your scoring: they would invoke either solution 1 or solution 2. Indeed, if a test-taker refused to answer the question, solution 3 would give them a legitimate basis to argue that their non-response ought to get full credit, too!

In summary, this analysis addresses the second part of the question by concluding that any of the following responses to question 2 should be marked correct because each of them are defensible: A, B, D, A and D, and nothing. No other response can be defended and therefore would not be correct.

Answer 3

whuber gives a great analysis where multiple answers are allowed. However, there is also a consistent way to understand the question such that there is only a single correct answer (though we need to state this as part of the question):

If you choose an answer to this question at random, what is the chance you will be correct, given that there is only a single correct answer?

A) 50% B) 25% C) 60% D) 50%

Again, we will define a "correct" answer as one that is rationally defensible and will follow the argument of whuber. A marking is a map from the set of answers to $\{0,1\}$ such that correct answers are sent to 1 and incorrect answers to 0. There are three possible self-consistent markings:

Solution 1                Solution 2                Solution 3
Ticket Answer Mark        Ticket Answer Mark        Ticket Answer Mark
     A    50%    1             A    50%    0             A    50%    0
     B    25%    0             B    25%    1             B    25%    0
     C    60%    0             C    60%    0             C    60%    0
     D    50%    1             D    50%    0             D    50%    0

Yet we need another logical step to narrow these down to a single defensible answer. When making this problem, the teacher was faced with three possible markings, each of which could be as equally an acceptable single answer as the others. However, as only one answer can be correct, the teacher should choose randomly between them. This assigns an equal probability to each marking so that:

• $1/3$ of the time the student will be right $50\%$ of the time
• $1/3$ of the time the student will be right $25\%$ of the time
• $1/3$ of the time the student will be right $0\%$ of the time

This results in the expectation that the student will be right $(50+25+0)/3=25$% of the time. Thus 25% should be the correct answer and the marking for Solution 2 should be chosen. This is a self-consistent update of the teacher's prior over the three possible markings, i.e. if Solution 2 is fixed to be the correct marking 100% of the time, then 25% is still the one correct answer.

Summary: If we specify that there is only a single correct answer, then that answer is 25%.

Answer 4

I believe the answer is 1/3. We don't know which answer (25%, 50%, or 60%) is correct. So, each answer, 25%, 50%, and 60% has a 1/3 chance of being correct if selected. Even though 25% appears twice, it still has a 1/3 chance of being the correct answer. It actually does not matter how many times 25% appears as an answer. If it appears 10 times along with the 50% and the 60%, the chance that it is the correct answer would still be 1/3.This assumes that one of the answers is correct. If there is a possibility of none of the answers being correct, then the answer would be 1/4. This is based on my interpretation of what the question is asking.

Answer 5

This multiple choice question isn't really a question. It's nonsense text which fails to communicate any meaning.

In order for it to communicate meaning, the meaning of the subconcept of "a correct answer to this question" would first have to be understood - but that is impossible without first understanding the meaning of the whole question. This creates an unresolvable circular dependency.

You might as well ask, "What is the probability that asdfaghauerg fhafufha?"

---

Consider this Python function:

def get_correct_answer():
  correct_answer = get_correct_answer()
  if correct_answer == 0.25:
    return 0.25
  if correct_answer == 0.5:
    return 0.5
  if correct_answer == 0.6:
    return 0.25
  return 0

One might argue that the only consistent values for the output of get_correct_answer are 0, 0.25, and 0.5, so the function must return one of those. But really the function just keeps calling itself recursively and never returns anything - which is the same thing that happens when you try to resolve this multiple choice question.