Understanding basic identifiability

Question

I'm having trouble understanding identifiability. Specifically, I'm not sure, in the following example, why $P\left(C\right)$ cannot be identified. Here's the example:

You have 2 unfair coins with the following probabilities:

$$P\left(H|C_1\right)=1/4$$ $$P\left(H|C_2\right)=3/4$$

Each coin has an equal probability of being chosen, so $P\left(C=C_1\right)=P\left(C=C_2\right)=1/2$.

My question is, given that I only observe $H$, why can't I identify $P\left(C\right)$ using basic algebra. That is, \begin{eqnarray} P\left(H\right)&=&P(C_1)P\left(H|C_1\right) + P\left(C_2\right)P\left(H|C_2\right)\\ &=& P\left(C\right)\left(P\left(H|C_1\right)+P\left(H|C_2\right)\right)\\ P\left(C\right) &=& \frac{P\left(H\right)}{P\left(H|C_1\right)+P\left(H|C_2\right)} \end{eqnarray}

In this case $P\left(C\right)$ would seem to be identifiable, so I'm not sure where I'm going wrong. Note that I'm not looking to identify $C$ itself (the actual coin), I want to know the distribution on $C$.

Answer 1

Identifiability problems arise in parameter estimation when you have two or more different sets of parameter values that give rise to the same probabilities of observing the random variable. In that case, you can't distinguish between the different possible values of the parameters by looking at the data, no matter how much data you have. (Writing loosely here...)

In your case, I suspect you're slightly misunderstanding what you observe and what you aren't. Typically you would observe the outcome of the coin flip $H$ or $T$, and you wouldn't know the probabilities, because if you did identifiability is a moot issue (since you've no parameters to estimate.)

In this case, let us assume that you DON'T know $P(H|C_1)$, $P(H|C_2)$, and $P(C_1)$, and all you observe is some collection of heads and tails. I assume you are interested in estimating the three probabilities.

Now, note that if $P(H|C_1)=0.75$, $P(H|C_2)=0.25$, and $P(C_1)=0.25$, then:

$P(H) = P(H|C_1)P(C_1) + P(H|C_2)P(C_2) = 0.75*0.25 + 0.25*0.75 = 6/16$

But if $P(H|C_1)=0.25$, $P(H|C_2)=0.75$, and $P(C_1)=0.75$, then, skipping the detail, $P(H) = 6/16$ again. In fact, there are an infinite number of parameter values (the probabilities) that will generate $P(H)=6/16$, or any other value you care to pick. $P(C_1)$ can be any probability, and you can find a $P(H|C_1)$ and $P(H|C_2)$ that will generate any specific $P(H)$. Consequently, no sequence of heads and tails will allow you to estimate $P(C_1)$. $P(C_1)$ is not identifiable.