5
$\begingroup$

I'm working on a problem involving Bayesian updating with a Beta prior, but the data I observe comes from a slightly complex source.

Let $X \sim \text{Bernoulli}(p)$ and $Y \sim \text{Bernoulli}(q)$, where $q$ is known. I don't directly observe $X$. Instead, I observe $Z = XY$. I would like to estimate $p$.

Here are some intuition based on some preliminary calculations:

  • If $Z = 1$, I believe the update on the prior $\text{Beta}(\alpha, \beta)$ should be $\text{Beta}(\alpha+1,\beta)$ since $X$ definitely equals 1.

  • If $Z = 0$, I deduced that the update should be $\text{Beta}(\alpha+P(X=1|Z=0),\beta+1−P(X=1|Z=0))$.

Can someone provide a detailed proof or justification for these updates? Specifically, I'm interested in the logic behind the fractional update when observing $Z = 0$ and its validity within the Bayesian framework.

$\endgroup$

1 Answer 1

6
$\begingroup$

Your intuition here is not quite accurate and the situation is a bit more complicated than the way you are treating it. Nevertheless, since the parameter $q$ is known, this problem is a simple variation of the standard beta-binomial model used widely in Bayesian analysis. Since $X \sim \text{Bernoulli}(p)$ and $Y \sim \text{Bernoulli}(q)$ you have:

$$Z = XY \sim \text{Bernoulli}(pq).$$

If we observe IID data $Z_1,...,Z_n$ from this distribution then we obtain the sufficient statistic:

$$\dot{Z}_n \equiv \sum_{i=1}^n Z_i \sim \text{Bin}(n, pq).$$

Taking the prior $p \sim \text{Beta}(\alpha, \beta)$ then gives the posterior kernel:

$$\begin{align} \pi(p|\mathbf{z}_n, q) &\propto \text{Bin}(\dot{z}_n | n, pq) \cdot \text{Beta}(p|\alpha, \beta) \\[6pt] &\propto (pq)^{\dot{z}_n} (1-pq)^{n-\dot{z}_n} p^{\alpha-1} (1-p)^{\beta-1}. \\[6pt] \end{align}$$

Consequently, the posterior density function is:

$$\pi(p|\mathbf{z}_n, q) = \frac{(pq)^{\dot{z}_n} (1-pq)^{n-\dot{z}_n} p^{\alpha-1} (1-p)^{\beta-1}}{\int_0^1 (pq)^{\dot{z}_n} (1-pq)^{n-\dot{z}_n} p^{\alpha-1} (1-p)^{\beta-1} \ dp}.$$

The scaling constant determined by the integral does not have a closed-form solution so you would need to evaluate it numerically. Alternatively you could simulate from the posterior distribution using MCMC methods or importance sampling methods.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.