How to find the probability of an unobserved binary variable from repeated noisy observations?

Question

Let $Y \in \{0,1\}; P(Y=1)=\beta$. We have no observations of $Y$.

Instead, we observe a sample of $A$,$B$. We can assume that $P(A,B|Y)=P(A|Y)P(B|Y)$; $P(A=Y)=P(B=Y)=\alpha$; and that $P(A=Y|Y)=P(A=Y)$, P(B=Y|Y)=P(B=Y). In other words, $A$ and $B$ are binary variables, are conditionally independent given $Y$, and have the same error rate and their errors are independent of $Y$.

$\alpha$ is unknown, but I have convinced myself it is easy to estimate since it is determined by $P(A \ne B)$.

How can we estimate $\beta$?

Based on a comment on page 352 of the book "Measurement Error in Nonlinear Models" by Carroll et. al 2006, I believe that it is possible to consistently estimate $\beta$. But I'm having trouble working it out. I also recall doing a homework problem like this in the past, but I don't recall where.

I have worked out that $P(A\ne B) = 2\alpha (1-\alpha)$.

Carroll et al. also suggest that with a sample of $A$,$B$,$C$ and $P(A,B,C|Y)=P(A|Y)P(B|Y)P(C|Y)$ we can estimate $\beta$ even if $P(A=Y) \ne P(B=Y) \ne P(C=Y)$.

EDIT: Prior versions of the question did not assume that error rates were independent of $Y$. This assumption seems important to make the problem solvable, at least in the case of two replicates.

Answer 1

I think the answer you provided has a mistake. You substitute $P(A=1|Y=1)$ as $\alpha$. This is wrong according to your definition of $\alpha$ above. $ \alpha = P(A=Y) = P(A=1, Y=1) + P(A=0, Y=0)$, no?

I haven't found the solution, but I have developed some intuition that may be helpful.

Note that the joint probability distribution of two binary variables $A$ and $Y$ where $P(A=1) = \alpha$ and $P(Y=1) = \beta$ can be written as:

	A=0	A=1
Y=0	$\bar\alpha\bar\beta + k$	$\alpha\bar\beta - k$
Y=1	$\bar\alpha\bar\beta - k$	$\alpha\beta + k$

Where $\bar\alpha = 1 - \alpha$, $\bar\beta = 1- \beta$, and $k$ controls the covariance between $A$ and $Y$:

$cov(A,Y) = E[AY] - E[A]E[Y] $

$cov(A,Y) = \overbrace{\alpha\beta + k}^{P(A=1, Y=1)} - \alpha\beta = k$

We can now calculate the covariance between $A$ and $B$ in the setup you proposed (where $P(B=1) = \alpha$) and find that:

$cov(A,B) = E[AB] - E[A]E[B] $

$cov(A,B) = E[AB] - \alpha^2 $

$cov(A,B) = P(A=1, B=1 | Y = 0) + P(A=1, B=1 | Y = 1) - \alpha^2 $

$cov(A,B) = P(A=1| Y = 0) P(B=1| Y = 0) +P(A=1| Y = 1) P(B=1| Y = 1) - \alpha^2 $

$cov(A,B) = P(A=1| Y = 0)^2 +P(A=1| Y = 1)^2 - \alpha^2 $

If we substitute this using the table above, we can find that:

$cov(A,B) = k^2 Var[Y]^{-1} = \frac{cov(A,Y)^2}{Var[Y]}$

Answer 2

I found a solution for the case where we have 2 repeated noisy observations. It is not hard to spot once you write out the joint probability, use it to write out the conditional probability table in terms of $\beta$ and $\alpha$. We can show that $P(A\ne B) = 2 \alpha(1-\alpha)$, then use the quadratic formula to solve for $\alpha$, and then solve for $\beta$ in terms of $\alpha$.

Joint probability table:

	$Y=0$	$Y=1$	$P(A)$
$A=0$	$\alpha(1-\beta)$	$(1-\alpha)\beta$	$\alpha(1-\beta) + (1-\alpha)(\beta)$
$A=1$	$(1-\alpha)(1-\beta)$	$\alpha\beta$	$(1-\alpha)(1-\beta)+\alpha\beta$
$P(Y)$	$\alpha(1-\beta) + (1-\alpha)(1-\beta)$	$(1-\alpha)\beta +\alpha\beta$	1

If you doubt that $P(A=1,Y=1)=\alpha\beta$, consider $P(A=1,Y=1)=P(A=Y,A=1,Y=1)=P(A=Y,Y=1)=P(A=Y)P(Y=1)$

The last equality seems surprising since it says that $A=Y$ and $Y=1$ are independent. But they are since $P(A=Y)=\alpha$ no matter the value of $Y$.

Conditional probability table:

	$Y=0$	$Y=1$
$P(A=0|Y)$	$\frac{(1-\alpha)(1-\beta)}{\alpha(1-\beta)+(1-\alpha)(1-\beta)}$	$\frac{(1-\alpha)\beta}{(1-\alpha)\beta+\alpha\beta}$
$P(A=1|Y)$	$\frac{(1-\alpha)(1-\beta)}{\alpha(1-\beta)+(1-\alpha)(1-\beta)}$	$\frac{\alpha\beta}{(1-\alpha)\beta + \alpha\beta}$

Lemma 1: $P(A=1|Y=1)=\alpha$; $P(A=0|Y=1)=(1-\alpha)$ Simplifying $P(A=1|Y=1)$ from the conditional probability table, we have: $$\frac{\alpha\beta}{(1-\alpha)\beta + \alpha \beta} = \frac{\alpha\beta}{\beta}=\alpha$$.

Simplifying $P(A=0|Y=1)$ we have: $$\frac{(1-\alpha)\beta}{(1-\alpha)\beta+\alpha\beta}=\frac{1-\alpha}{\alpha+1-\alpha}=1-\alpha$$

Lemma 2: $P(A \ne B) = 2 \alpha(1-\alpha)$ Consider the case $P(A\ne B,Y=1)$. The other is similar. $$P(A \ne B,Y=1) \\ = P(A\ne B|Y=1)P(Y=1) \\= P(Y=1)(P(A=1,B=0)|Y=1) + P(A=0,B=1|Y=1)) \\= P(Y=1)(P(A=1|Y=1)P(B=0|Y=1)+P(A=0|Y=1)P(B=1|Y=1)) \\ = \beta(\alpha(1-\alpha) + (1-\alpha)\alpha) = 2 \beta \alpha (1-\alpha)$$

since $A$ and $B$ are conditionally independent given Y.

Clearly, $P(A\ne B,Y=0) = 2 (1-\beta) \alpha (1 - \alpha)$. So we can write $$P(A \ne B) = 2 \alpha (1-\alpha)\beta + 2 \alpha (1-\alpha)(1-\beta) = 2\alpha(1-\alpha)(\beta + 1 - \beta) = 2\alpha(1-\alpha)$$

Use the quadratic formula to solve for $\alpha$. $$2\alpha^2 - 2\alpha + P(A\ne B) = 0 $$

$$ \alpha = \frac{2 \pm \sqrt{4 - 8P(A \ne B)}}{4} $$

Finally use $\alpha$ to solve for $\beta$

Observe that $P(Y=1|A\ne B) = P(Y=1)$.

$$ P(Y=1|A\ne B) = \frac{P(A\ne B | Y=1)P(Y=1)}{P(A \ne B)}$$ $$P(A \ne B | Y=1) = P(A=1,B=0 | Y=1) + P(A=0,B=1|Y=1)$$ $$ = 2 P(A=1,B=0 | Y=1) = 2P(A=1|Y=1)P(B=0|Y=1) = 2P(A=1|Y=1)P(A=0|Y=1) = 2P(A=1|Y=1)(1-P(A=1|Y=1) = $$

$$ \frac{P(A=1,B=1) - \alpha ^2 + 2\alpha -1}{2\alpha - 1} = \beta $$

This is because $$P(A=1,B=1) = P(A=1,B=1,Y=1) + P(A = 1, B = 1, Y = 0)$$ Consider $P(A=1,B=1,Y=1)$.

$$P(A=1,B=1,Y=1)=P(A=1, B=1|Y=1)P(Y=1) = P(A=1|Y=1)P(B=1|Y=1)P(Y=1)=\alpha^2\beta$$

And similarly, $P(A=1,B=1,Y=0) = (1-\alpha)^2(1-\beta)$. Therefore $$P(A=1,B=1) = \alpha^2\beta + (1-\alpha)^2(1-\beta) = \beta(\alpha ^ 2 -(1-\alpha)^2) + (1-\alpha)^2$$

And therefore $$\beta = \frac{P(A=1,B=1) - (1-\alpha)^2}{\alpha^2 - (1-\alpha)^2}$$

So it's clear that if we can estimate $P(A=1,B=1)$ and $P(A\ne B)$ we have an estimator of $P(Y)$.

The use of the quadratic formula deserves some attention. If $P (A \ne B) > 0.5$ then $\alpha$ is not real. This suggests that $P(A \ne B) <= 0.5$ by construction. Indeed, $P(A\ne B) > 0.5$ violates the conditional independence of $A$ and $B$ given $Y$ by requiring $A$ and $B$ to be anti-correlated. It is also possible that there are 2 real solutions for $\alpha$. One will have $\alpha > 0.5$ the other will have $\alpha < 0.5$, but only one of these will give a valid solution for $\beta$.