2
$\begingroup$

Normally, when we have $$p\sim Beta(a,b)$$ and an observation of $x=1$ (''success'') from a Bernoulli trial with ''success'' probability $p$, the Bayesian inference on the parameter value $p$ is $$p|x\sim Beta(a+1,b).$$ And the expectation about the next Bernoulli trial becomes $$\frac{a+1}{a+b+1}.$$ What I'm curious for myself is what happens if we only observe noisy outcomes?

For example, when the actual outcome is ''failure'', we sometimes observe ''success''. Say this kind of error happens with probability $0.3$ when ''failure'' happens. So, when we observe ''success'', it may not really mean the real outcome is ''success''. On the other hand, if we observe ''failure'', it's an accurate observation as ''success'' always produces ''success'' signal.

If this is the case, what should be the correct Bayesian inference after observing a ''success''?

$\endgroup$
4
  • $\begingroup$ Express your noisy outcome as a likelihood and multiply by the density of the beta prior. Or simulate, roughly as here. $\endgroup$
    – BruceET
    Jul 25, 2019 at 16:21
  • $\begingroup$ Are you assuming you know the false positive and false negative probabilities? $\endgroup$
    – jbowman
    Jul 25, 2019 at 18:22
  • $\begingroup$ @BruceET - I don't think this will work, because the parameters won't be identifiable, e.g., $p = 0.3$ with false positive and false negative probabilties = 0 is indistinguishable from $p=0.7$ with false positive and false negative probabilities = 1. I haven't fully thought this through, but that's my strong impression. $\endgroup$
    – jbowman
    Jul 25, 2019 at 18:33
  • $\begingroup$ @jbowman - Thanks for your comment. I'm thinking of a situation when the probabilities of false positive / false negative are fixed constants. $\endgroup$
    – Andeanlll
    Jul 26, 2019 at 0:45

1 Answer 1

5
+50
$\begingroup$

My answer follows the suggestion to express the noisy outcome as a likelihood. I have changed the notation a bit (from the question) to handle some additional complications.

Let \begin{equation} p(y_t|\theta) = \textsf{Bernoulli}(\theta) = \begin{cases} \theta & y_t = 1 \\ 1-\theta & y_t = 0 \end{cases} , \end{equation} where \begin{equation} p(\theta) = \textsf{Beta}(\theta|a,b) . \end{equation} Suppose we observe $y_1$. Then, as indicated in the setup to the question, \begin{equation} p(\theta|y_1) = \textsf{Beta}(\theta|a+y_1,b+1-y_1) \end{equation} and \begin{equation} p(y_2|y_1) = \textsf{Bernoulli}\left(y_2\Big|\frac{a+y_1}{a+b+1}\right) . \end{equation}

Now suppose we don't observe $y_t$ directly. Instead we observe a noisy report, $z_t$, where \begin{equation} p(z_t|y_t) = \textsf{Bernoulli}(z_t|q_{y_t}) = \begin{cases} q_{y_t} & z_t = 1 \\ 1-q_{y_t} & z_t = 0 \end{cases} . \end{equation} In the question (as I understand it), $q_0 = .7$ and $q_1 = 1$.

We now have a complete model, which is to say we have the following joint distribution: \begin{equation} p(y_t,z_t,\theta) = p(z_t|y_t)\,p(y_t|\theta)\,p(\theta) , \end{equation} where $(a,b,q_0,q_1)$ are known. Given this model, we have \begin{equation} p(y_t,\theta|z_t) = \frac{p(y_t,z_t,\theta)}{p(z_t)} , \end{equation} where \begin{equation} p(z_t) = \sum_{y_t\in\{0,1\}} p(z_t|y_t)\,\int p(y_t|\theta)\,p(\theta)\,d\theta = \textsf{Bernoulli}\left(z_t\Big|\frac{a\,q_1+b\,q_0}{a+b}\right) . \end{equation} There are a number of ways one can proceed at this point to estimate the model (i.e., the compute the posterior distribution).

In addition there are a number of distributions we can address. Suppose we observe $z_1$. In addition to $p(\theta|z_1)$, we have \begin{equation} p(y_1|z_1) \qquad\text{and}\qquad p(y_t|z_1) , \end{equation} where $t \ge 2$. The first of these two distributions is specific to $y_1$ since it is based on its own signal, while the second is generic since is applies to any $y_t$ for which we as yet have no signal. For $t \ge 2$, note \begin{equation} p(y_t|z_1) = \int p(y_t|\theta)\,p(\theta|z_1)\,d\theta . \end{equation}

It is straightforward to extend this approach to allow for multiple observations where $y_{1:T} = (y_1, \ldots, y_T)$, $z_{1:T} = (z_1,\ldots,z_T)$, and \begin{equation} p(y_{1:T},z_{1:T},\theta) = p(z_{1:T}|y_{1:T})\,p(y_{1:T}|\theta)\,p(\theta) = \left(\prod_{t=1}^T p(z_t|y_t)\,p(y_t|\theta)\right) p(\theta) . \end{equation} We can compute a number of posterior distributions including $p(\theta|z_{1:T})$ as well as the specific distributions $p(y_t|z_{1:T})$ for $1 \le t \le T$ and the generic distribution \begin{equation} p(y_{T+1}|z_{1:T}) = \int p(y_{T+1}|\theta)\,p(\theta|z_{1:T})\,d\theta . \end{equation}

$\endgroup$
3
  • $\begingroup$ Thank you very much for your clear explanation. It's well written and easy to understand! $\endgroup$
    – Andeanlll
    Jul 26, 2019 at 15:11
  • $\begingroup$ You are quite welcome. I'm glad you found it useful. $\endgroup$
    – mef
    Jul 26, 2019 at 15:12
  • $\begingroup$ Superb derivation! One tiny correction?, you meant, in op's example, $q_0 = p(z = 1 | y = 0) = 0.3$, rather than 0.7? $\endgroup$ Jun 11, 2020 at 23:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.