1
$\begingroup$

I’m trying to determine the best way to detect if a coin is unbiased, given some desired alpha. I understand basic probability/statistical inferencing, but there’s some information out there that confuses me.

In particular, I was reading the wikipedia article on this checking whether coin is fair. For the Bayesian approach, the wikipedia article indicates “The probability for an unbiased coin (defined for this purpose as one whose probability of coming down heads is somewhere between 45% and 55%)”. This seems kind of arbitrary. Here’s how I would have done this (and then I want help comparing with the wikipedia approach):

  1. Choose an $\alpha$ (probably $\alpha=0.05$)

  2. Simulate the coin in question using bootstrapping to estimate the sample mean. Each bootstrap sample is a binomial distribution. (assume some number of tosses “$n$” for each bootstrap sample). Compute “$N$” binomial distributions to get $N$ bootstrap samples.

  3. By central limit theorem, the distribution of the sample means will be Gaussian

  4. Because sample mean and sample variance are unbiased statistics, this Gaussian distribution will be parameterized by:

    • mean of sample means

    • mean of the sample variances

  5. Compute the confidence interval for this Gaussian distribution of the sample mean based on the given $\alpha=0.05$.

  6. If 0.5 falls within that confidence interval, then coin is unbiased, otherwise biased.

I have a few questions about my approach:

  • Does the desired alpha affect “n” and/or “N” in my bootstrap sampling? If so, how? If not, then how to choose these numbers?

  • It doesn’t seem necessary to simulate the unbiased coin, since we are testing the null hypothesis that the $\mu=0.5$, and the variability in an unbiased coin is captured in the confidence interval. Correct?

Related: How to reconcile the frequentist method with assessing the fairness of a coin?

$\endgroup$
1
  • $\begingroup$ I have a follow-up question about how to use the terminology. this approach seems close to a frequentist approach in my opinion, since we’re simulating the results of an unbiased/biased. Is it a frequentist approach? If not, then what approach is this closest to? $\endgroup$
    – makansij
    Feb 16, 2020 at 16:39

1 Answer 1

2
$\begingroup$

All coins are biased. It's a question of how biased they are, but no coin has exactly equal chances of being heads or tails.

In the frequentist approach, I think this calls for a test of equivalence:

  1. Decide how close to 0.50000 you will accept as unbiased.
  2. Do a power analysis to determine how many flips you will need to have a good (0.80? 0.90?) chance of detecting that level of bias.
  3. Flip the coin that many times
  4. Do a test of equivalence.
$\endgroup$
8
  • 1
    $\begingroup$ Because we use a "coin" as a model for much more important phenomena, it seems unjustifiable (or irrelevant) to claim "all" coins are biased, because that amounts to asserting everything is associated with everything else. Ordinarily the question is framed as "is it possible to detect a bias of a specified size with a specified chance" and that leads to standard frequentist tests, not equivalence tests. $\endgroup$
    – whuber
    Feb 17, 2020 at 14:01
  • 1
    $\begingroup$ Worth noting - Step 1 here is just as arbitrary as the one used in the Bayesian approach. $\endgroup$
    – jkm
    Feb 17, 2020 at 14:01
  • 1
    $\begingroup$ @jkm That arbitrariness goes away in a decision-theoretic setting, which provides guidance for setting the degree of closeness in terms of loss and risk. $\endgroup$
    – whuber
    Feb 17, 2020 at 14:02
  • 2
    $\begingroup$ @whuber I'm not against it being arbitrary, the OP originally did. I can't see any 'objective', one-size-fits-all measure of how fair is fair even being possible. Both approaches ultimately tailor the criteria to a concrete real-world purpose (or at least should). $\endgroup$
    – jkm
    Feb 17, 2020 at 14:07
  • 1
    $\begingroup$ @jkm Thank you for sharing that nuance--I agree with it, incidentally. $\endgroup$
    – whuber
    Feb 17, 2020 at 14:11

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.