# Check whether a coin is fair

An interview problem is like the following:

Given a coin you don’t know it’s fair or unfair. Throw it 6 times and get 1 tail and 5 head. Determine whether it’s fair or not. What’s your confidence value?

I came out the following solution:

$H_0:$ the coin is fair $H_a:$ the coin is unfair

$X$: is the number of heads

Rejection region: $|X - 3| > 2$, i.e., $X = 0,1,5,6$

Significance level alpha:

$\alpha = P(\text{reject }H_0 \mid H_0 \text{ is true})$ $=P(X=0,1,5,6 \mid H_0 \text{ is true})$ $= (\binom{6}{0}+\binom{6}{1}+\binom{6}{5}+\binom{6}{6})*(1/2)^6$ $= (1+6+6+1)*(0.5^6) = 0.21875$

because $\alpha > 0.05$, we do not have enough evidence to reject $H_0$, and we accept $H_0$, so the coin is fair

Is the above test a good one? And I did not know how to calculate the confidence value?

• @Tim, no, it's an interview question – FihopZz Sep 7 '15 at 14:53
• It still falls into self-study category, see stats.stackexchange.com/tags/self-study/info – Tim Sep 7 '15 at 14:54
• While I spend a lot of text explaining why a hypothesis test won't answer the question that was asked and indeed why no amount of data can be used to demonstrate that the coin is actually fair, I've added a section at the end explaining several things wrong in your attempted hypothesis test. (If you actually said those things in the interview, I expect you didn't get the job, because some of those errors are fundamental.) – Glen_b -Reinstate Monica Sep 8 '15 at 23:55

[I think I'd start by asking for a whiteboard, markers -- and an eraser, because one boardful isn't enough to explain everything wrong with the question.]

I'm going to answer this question by rejecting its premises.

1. The "coin" itself is just a coin; by itself it doesn't do anything, and so it cannot be fair or not-fair. What we're talking about is the process of tossing a particular coin in some fashion -- that can be discussed in terms of whether it's fair or not.

2. Data can't show you that a coin-tossing process applied to some coin is exactly fair. Sometimes it can show you that your coin-tossing-process on a given coin is inconsistent with fairness, but failure to identify any inconsistency with fairness doesn't imply fairness (failure to reject is because your sample size is small, not because the coin is actually fair).

[e.g. Consider it in terms of a confidence interval for P(head), the fact that $\frac12$ is in the CI doesn't mean that P(head)=$\frac12$, since there are always other values - distinct from $\frac12$ - in there too. Or think in terms of power: on the experiment given in the interview question - 6 tosses - what's the probability that you'd reject as unfair the case where the tossing process applied to a particular coin had $p(\text{head})=0.51$ at some typical significance level? That's clearly an unfair coin, but you'll reject barely more often than your type I error rate, and a large fraction of those rejections in a two tailed test would be "in the wrong tail"!]

3. No coin-tossing process on a given coin will be perfectly fair. (For example, changing the side facing up slightly alters the chances associated with the resulting face on the toss, as experiments run by Persi Diaconis have shown.)

Could the coin be close to fair? Possibly; it may even be possible to get very close to fair. Exactly fair? No, it's not possible in practice. But then to discuss whether it's "close to fair" we'd have to define what we mean by 'close'. [If we were to give some usable definition, while some people might suggest some form of equivalence test, or perhaps considering whether some CI lay entirely inside some "close to fair" bounds, I'd be inclined toward a Bayesian approach to deciding whether the coin is sufficiently close to fair. Note that with the tiny sample size mentioned, the data are quite consistent with p(head) so far from $\frac{1}{2}$ that this exercise on that data would not conclude "close to fair" on any of the three mentioned approaches.]

So:

Given a coin you don’t know it’s fair or unfair.

Yes, actually, I do. In fact I don't even need to see data. It's not fair.

Throw it 6 times and get 1 tail and 5 heads. Determine whether it’s fair or not.

I really don't care what the data are. It makes no difference to my answer, since the data could not possibly demonstrate fairness, even if fairness were a realistically possible state to be in.

100% (in a sense similar to almost surely)

(In any case, even if there were a way to do this statistically I don't know of any statistical procedure that gives anything I'd agree to call "confidence values", so I also reject the form of that question. What does that term even mean? If I were asked a question phrased that way in an interview, I'd have serious concerns about working there, because it seems to suggest the people conducting the interview don't really understand what they're even asking - and that suggests either nobody there knows this stuff, or they don't care enough about this position to make sure the interview is being conducted by someone who does. Either way, it would certainly influence my willingness to work there.)

Forgetting everything I just said for the moment, some comments on your hypothesis test:

Your process for a hypothesis test is wrong.

1. Why do you compare your significance level with 0.05? You've chosen a significance level of 0.21 (which I have no objection to in this experiment, the sample size is so low you only have 3% or 21% and $\alpha$=3% will be too low-powered to be much use) -- 0.05 doesn't relate to anything here.

2. Do you see that in your test when it came time to reject or not reject, you made no reference at all to the sample statistic (5 heads)? Indeed you ignored your rejection rule.

3. The rejection rule you stated algebraically $|X-3|>2$ is inconsistent with the rejection region you mentioned ($0,1,5,6$).

That's a lot of errors in a few lines! If I was involved in such an interview**, I might forgive the error with the rejection rule as something one could overlook under interview pressure, but the first two errors would suggest some fundamental problems.

** leaving aside that I'd never ask such a poor question, nor would I likely care enough about hypothesis testing to even think to ask a question about it.

• As you know I take your posts as free tuition, and read them with attention. That's why I want to ask you if elaborating on possible Bayesian prior assumptions, and resulting posterior pdf's as in this code I compile from a youtube quoted in the github file would be meritorious. – Antoni Parellada Sep 8 '15 at 17:30
• @Antoni Are you asking if you should post an answer to that effect? If so, I'd say "yes, there might be value in that" (but keeping in mind generally people will get value from plots and an outline of the approach rather than from reading the code; you'll still at least want to point to code of course, or include it if it's short enough). ... ctd – Glen_b -Reinstate Monica Sep 8 '15 at 23:18
• ctd... On the other hand, if you're asking if I'd like to change my answer to do so, I don't think so, at least not at this point, but I'd think about it (for that second option, however, I have added a little to my parenthetic comment after item 3, to explain why on the data in the question the result is obvious for the "is the coin close to fair" question - we simply can't conclude that it is.] – Glen_b -Reinstate Monica Sep 8 '15 at 23:28
• You read me well on your first reply. I was thinking of writing something when I read "bayesian" on your original answer, which was very clear from the beginning, but reminded me of the source I quoted. I didn't know if something along the lines of the code on github made sense. I'll work on it, and see what comes out. Thank you! – Antoni Parellada Sep 9 '15 at 1:21
• +1 on this answer for the middle section. I don't understand what the first point is about? Doesn't calling a coin fair mean that its bias is equal to (or is close to) $\frac12$? – Neil G Sep 9 '15 at 6:34

I am not going to attempt to provide final answers to your question; I believe the topic is more than addressed after the comprehensive response given by Glen. However, and apropos of his comment about a Bayesian approach, I'd like to post some illustrations about the way our preconceptions about the "fairness" of the coin (or the experiment in general) affects the posterior probability density, i.e. the $p\,(\theta\,\vert\,\text{Data})$, where $\theta$ stands for the probability of heads in the coin toss.

Luckily, we have a conjugate prior distribution for the binomial case that occupies us - the beta distribution, facilitating the calculation of the posterior distribution.

First scenario - The Fair-Minded Player:


We walk into the game (not a very exciting game, but still...), and we have absolutely no reason to assume that there is foul play going on. Things being by nature less than perfect, we have it in our mind that the coin is fair-ish. In other words, we think that the probability of heads, $\theta$, falls somewhere around $\frac{1}{2}$. Later, the unexpected single tail out of $6$ tosses, will force us to move the posterior probability of $\theta$ to the left (the arrows indicate the influence of the data on the prior distribution):

Second Scenario - The Shrewd Player:


We strongly suspect from insider's leaked information that the game is markedly biased towards tails, and we not only are about to make a killing, but also in need to further reinforce our conviction after the first round, doubling down our bet:

Third Scenario - Losing Your Shirt:


We've never played before, but we have read a manual, and we feel ready. All signs clearly indicate that the coin is markedly biased towards $heads$, a mistake that we will soon start to correct at a high $\$\$cost: Fourth Scenario - No Idea Whatsoever:  It's a good thing that the$\beta(1,1)$distribution turns into a$U\,(0,1)$to address this scenario, where only the likelihood will influence the -posterior probability of$\theta$. As brought up to my attention, a Jeffreys prior is close and possibly more correct: So I hope this provides a bit of a light-hearted visual depiction of our approach to estimating the chances of this game being rigged, perhaps encapsulating more of a real scenario than calculations of the type pbinom(1, 6, 0.5). If you want the code in R, and the credits to a great video with Matlab illustrations, I posted it here. • No idea whatsover is the Jeffreys prior — not the uniform prior. – Neil G Sep 9 '15 at 6:23 • @NeilG Can you help me come up with a better term for the uniform? Ty btw – Antoni Parellada Sep 9 '15 at 6:29 • Use a Beta distribution with both parameters equal to$\frac12$. If you're interested in exploring this topic on your own, you can try to draw all the same graphs you drew in a different parametrization. For example, assume that you've landed on an alien planet who think of probability on a$\ell \in (-\infty, \infty)$scale whereby$\ell = \log \frac{p}{1-p}\$. You'll find that if you use an improper uniform prior in that space, you will end up with a different maximum likelihood value. – Neil G Sep 9 '15 at 6:32
• (my apologies, a different maximum a posteriori value). – Neil G Sep 9 '15 at 6:38