How to figure out whether a coin is "weighted" with some number of flips

Question

Suppose there's a weighted coin. That coin either lands on heads every 1/10 times it is tossed, or it never lands on heads at all. I don't know whether that coin is the type of coin that lands on heads 1/10 of the time, or if it's the type of coin that never lands on heads, but I want to figure this out by tossing the coin many times.

Suppose I toss the coin 20 times and it's Tails every time. What's the probability the coin is the type of coin that never lands on heads? Generally, how would I figure this kind of thing out for some other number of times, or some other "coin flip" probability?

Edit: this isn't a homework problem. I'm just used to thinking in "homework problem" language since I don't normally do this kind of math day to day.

I'm a software engineer trying to get rid of an intermittent issue that happens somewhere around 1/10 of the time. I want to be able to say I fixed it with, say, 95% confidence after running a test some number of times. It's an expensive test to run, so I don't want to run it more than I have to to be nearly certain it's fixed.

Answer 1

Let's call head probability as $p$ as usually in literature, and call the event $n$ tails as $A$ for short notation. Probability of having $n$ tails, given a $p$ (i.e. when you know it) is: $P(A|p)=(1-p)^{n}$. This is called the likelihood. When $p=0$, i.e. the coin never lands on heads, this is going to be $1$. But, we are actually interested in $P(p=0|A)$, i.e. the probability of the coin being the coin that never lands on heads given the fact that it is tossed $n$ times and never landed on head. Via Bayes Rule, we have

$$P(p=0|A)=\frac{P(A|p=0)P(p=0)}{P(A)}$$

Via Total Probability, we have: $P(A)=P(A|p=0)P(p=0)+P(A|p=1/10)P(p=1/10)$

Since you've only the options $p=0,1/10$, $P(p=1/10)=1-P(p=0)$. The only unknown remained here is $P(p=0)$, called the prior belief, which is not available for the most of the time, and needs to be assumed! For example, if you're very confident that you actually solved your bug, then set $P(p=0)$ to a high value such as $0.9$ or so. A typical approach is setting the prior to $1/2$, i.e. equal priors. In this case, your equation becomes:

$$P(p=0|A)=\frac{P(A|p=0)}{P(A|p=0)+P(A|p=1/10)}=\frac{1}{1+\left(\frac{9}{10}\right)^{n}}$$

If you want to be as confident as $0.95$ under this prior assumption, you need to set $n\approx 28$. Or, plot the curve of it and choose your confidence level.