4
$\begingroup$

My first and I think naive question here.

I am trying to model a certain business, and the simplest model I am willing to test is: 1. there is a bag of differently biased coins. 2. every step, a single coin is chosen with equal probability. 3. the chosen coin is flipped and returned to the bag.

The business goal is to predict the rate of heads in future trials (probably using Bayesian inference).

This got me thinking - am I not over-complicating things? Isn't this process (observationally) equivalent to a single biased coin?

Thanks in advance!

Improve this question
$\endgroup$

2 Answers 2

Reset to default
4
$\begingroup$

Yes, this process is equivalent to flipping a single biased coin.

In practice, there is often room to change the procedure, in which case there may be differences. For example, if after you choose a coin, you get to observe multiple flips from that coin rather than just one, then the distribution of biases matters. A bag with a $0\%$ coin and a $100\%$ coin will never produce the sequence HT (with the altered procedure), whereas a bag with a $50\%$ coin may.

Improve this answer
$\endgroup$
5
  • $\begingroup$ Thanks for confirmation! In fact, it might be possible that I will be able to refine the data gathering procedure to assign identifiers to coins - in this way it will become closer to what you said. $\endgroup$
    – Andris Birkmanis
    Feb 7, 2013 at 12:13
  • $\begingroup$ BTW, how would I go and prove the original process is equivalent to a single coin? Is the fact that for every possible binomial distribution there is a biased coin that results in it the proof? $\endgroup$
    – Andris Birkmanis
    Feb 7, 2013 at 12:23
  • 1
    $\begingroup$ This is just a check of the definitions. They have equal distributions. Random variables which take the values $0$ and $1$ are parametrized by the probability with which they take the value $1$. These are called Bernoulli random variables, not binomial random variables. Binomial random variables are something different, the number of heads when you flip several identical and independent coins with a fixed probability of heads. $\endgroup$
    – Douglas Zare
    Feb 7, 2013 at 13:03
  • $\begingroup$ But this reasoning is applicable only as long as individual trials are independent? Otherwise, the process is not equivalent to a sequence of Bernoulli trials? I used "binomial" because technically I receive data about big batches of trials, not individual ones. $\endgroup$
    – Andris Birkmanis
    Feb 7, 2013 at 13:14
  • $\begingroup$ Right, to get a sequence of independent Bernoulli trials, you choose which coin to toss independently. If the coins are related the Bernoulli trials may not be independent. $\endgroup$
    – Douglas Zare
    Feb 7, 2013 at 14:32
1
$\begingroup$

This is a standard example illustrating the notion that conditional independence does not imply unconditional independence. If the experiment consists of choosing a coin at random and then tossing it $n > 1$ times, then the results of the $n$ tosses are conditionally independent observations of a $\text{Bernoulli}(p)$ random variable where $p$ is the parameter for the coin chosen, and thus the total number of Heads is (conditionally) a $\text{Binomial}(n,p)$ random variable. But, the $n$ tosses are not unconditionally independent.

More explicitly, with two coins with $P(\text{Heads}) = p_i$ and $n=2$, the conditional probabilities of a Head on each trial is $p_i$ and the two tosses are conditionally independent. But, the law of total probability says that $$P(\text{Heads}) = \frac{p_1+p_2}{2}$$ on each toss, whereas for the two trials, it also says that $$P(2 ~\text{Heads on}~2~\text{tosses}) = \frac{p_1^2+p_2^2}{2} \neq \left(\frac{p_1+p_2}{2}\right)^2$$ showing that the two tosses are not unconditionally independent.

In short, if the experiment consists of choosing a fresh coin for each toss, then the trials are independent and the result is equivalent to tossing a coin with probability $\bar{p}$ where $\bar{p}$ is the population mean, but this is not the case if the experiment consists of choosing a coin and tossing it several times. If the OP is receiving data about "big batches of trials, not individual ones" it is important to look carefully at the conditions of the experiment that resulted in the big batches of data.

Improve this answer
$\endgroup$
1
  • $\begingroup$ The batches are the artifact of the reporting, in fact every trial is independent (the coin is put back to bag after one toss). $\endgroup$
    – Andris Birkmanis
    Feb 7, 2013 at 19:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.