4
$\begingroup$

Suppose there's a weighted coin. That coin either lands on heads every 1/10 times it is tossed, or it never lands on heads at all. I don't know whether that coin is the type of coin that lands on heads 1/10 of the time, or if it's the type of coin that never lands on heads, but I want to figure this out by tossing the coin many times.

Suppose I toss the coin 20 times and it's Tails every time. What's the probability the coin is the type of coin that never lands on heads? Generally, how would I figure this kind of thing out for some other number of times, or some other "coin flip" probability?

Edit: this isn't a homework problem. I'm just used to thinking in "homework problem" language since I don't normally do this kind of math day to day.

I'm a software engineer trying to get rid of an intermittent issue that happens somewhere around 1/10 of the time. I want to be able to say I fixed it with, say, 95% confidence after running a test some number of times. It's an expensive test to run, so I don't want to run it more than I have to to be nearly certain it's fixed.

$\endgroup$
5
  • $\begingroup$ Welcome to Cross Validated, Andre! :) Since the problem seems like a homework problem, you should add the self-study tag and edit the question to describe how far you got in trying to solve it on your own. $\endgroup$ Mar 20, 2019 at 4:25
  • $\begingroup$ Thanks for the warm welcome! I edited my question to explain that it's not a homework problem. $\endgroup$
    – Andre
    Mar 20, 2019 at 5:23
  • $\begingroup$ You cannot produce a probability as an answer unless you also posit a prior probability. Your problem might not admit such an assumption, but it does have part of a loss function: namely, the cumulative cost of tests. In order to solve it you need to complete your description: please tell us what it would cost you to make the wrong decision. Only then can anyone give you a truly useful and correct answer--anything else risks being (far) too expensive to you. $\endgroup$
    – whuber
    Mar 20, 2019 at 12:18
  • $\begingroup$ Hey whuber, I think you might have misunderstood based on my edit. I'm not looking for someone to tell me how many tests I should run, just how to calculate the probability. I added that part just because Peter Leopold thought this was homework -- thanks! $\endgroup$
    – Andre
    Mar 21, 2019 at 1:06
  • $\begingroup$ I found your remark "It's an expensive test to run, so I don't want to run it more than I have to" to be particularly interesting and important. If that's the case, then asking how to achieve a certain confidence level misses the mark and only prompts replies that may turn out to be very costly for you. $\endgroup$
    – whuber
    Mar 21, 2019 at 14:44

1 Answer 1

3
$\begingroup$

Let's call head probability as $p$ as usually in literature, and call the event $n$ tails as $A$ for short notation. Probability of having $n$ tails, given a $p$ (i.e. when you know it) is: $P(A|p)=(1-p)^{n}$. This is called the likelihood. When $p=0$, i.e. the coin never lands on heads, this is going to be $1$. But, we are actually interested in $P(p=0|A)$, i.e. the probability of the coin being the coin that never lands on heads given the fact that it is tossed $n$ times and never landed on head. Via Bayes Rule, we have

$$P(p=0|A)=\frac{P(A|p=0)P(p=0)}{P(A)}$$

Via Total Probability, we have: $P(A)=P(A|p=0)P(p=0)+P(A|p=1/10)P(p=1/10)$

Since you've only the options $p=0,1/10$, $P(p=1/10)=1-P(p=0)$. The only unknown remained here is $P(p=0)$, called the prior belief, which is not available for the most of the time, and needs to be assumed! For example, if you're very confident that you actually solved your bug, then set $P(p=0)$ to a high value such as $0.9$ or so. A typical approach is setting the prior to $1/2$, i.e. equal priors. In this case, your equation becomes:

$$P(p=0|A)=\frac{P(A|p=0)}{P(A|p=0)+P(A|p=1/10)}=\frac{1}{1+\left(\frac{9}{10}\right)^{n}}$$

If you want to be as confident as $0.95$ under this prior assumption, you need to set $n\approx 28$. Or, plot the curve of it and choose your confidence level.

$\endgroup$
2
  • $\begingroup$ In a comment to the question I have noted that this answer risks being a terrible one, depending on what the actual cost function is. The problem is that it pays no attention to the OP's actual objectives, which is to optimize the cost of making this decision. $\endgroup$
    – whuber
    Mar 20, 2019 at 12:19
  • $\begingroup$ I think this is a great answer, and exactly what I was looking for. Accepted -- thanks gunes! $\endgroup$
    – Andre
    Mar 21, 2019 at 1:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.