The mean is affected by outliers in the distribution, so does not inform us about the mid-point of the distribution. The median is not affected by outliers.
Consider three example distributions and see how the presence of one outlier affects the mean but not the median:
||[1, 2, 3, 4, 5]
||[1, 2, 3, 4, 10]
||[1, 2, 3, 4, 20]
So relating these examples back to your interview question: assume A picks the median, so 50% of the time the sample will be less than A's guess and 50% of the time greater. Therefore, if B picks a different value from A, A's guess must be closer to the sample value at least 50% of the time.
Now assume B picks the mean. In the first case, they pick the same number, so the outcome is a draw. In the second case, A wins 60% of the time, B wins 20% of the time and it's a draw 20% of the time. In the third case, A wins 80% of the time and B wins 20% of the time.