2
$\begingroup$

I have a Beta-Bernoulli process $\textbf{X}=\{X_1, X_2...X_n\}$, $\textbf{p}=\{p_1, p_2...p_n\}$ where: $$X_i \sim Bernoulli(p_i)$$ $$p_i \sim Beta(\alpha,\beta)$$

where $\alpha$ and $\beta$ are known.

Is there an analytical solution for distribution of $p'$ for the subset of trials that were successful: $f_{p'|X_i=1}$

It's easier to illustrate in a simulation setting. That is, given sequence $\textbf{X}$, let $\textbf{X'}$ be the subset of $\textbf{X'}$ containing successes only: $$\textbf{X}'=\{X_i|X_i = 1\} = \{1,1,1...1\}$$

What is the distribution of $p$ that generated $\textbf{X'}$?

For example, below is the result from a simulation. The blue line is the distribution of $p$ from all trials ($p\sim Beta(2,2)$). Orange line is the conditional distribution of $p$ from successes only. Green line is the distribution of $p$ from failures only. So, the overall distribution of $p$ is a mixture of the two.

enter image description here

$\endgroup$
0
3
$\begingroup$

I suppose this is supposed to be a Bayesian question

If the prior for $p$ is $\operatorname{Beta}(\alpha, \beta)$ with density proportional to $p^{\alpha-1}(1-p)^{\beta-1}$ and you then observe $k$ successes and $n-k$ failures in $\mathbf{X}$ with likelihood proportional to $p^k(1-p)^{n-k}$

then the posterior density for $p$ given $\mathbf{X}$ is proportional to $p^{\alpha-1+ k}(1-p)^{\beta-1+ n-k}$ so the posterior distribution for $p$ is $\operatorname{Beta}(\alpha+k, \beta+n-k)$

If $\mathbf{X}$ is a single observation and is a success then this mean the posterior distribution for $p$ is $\operatorname{Beta}(\alpha+1, \beta)$


I now believe you think you are asking a different question, where different values of $p$ are drawn from $\operatorname{Beta}(\alpha, \beta)$ giving $p_1, p_2, \ldots, p_n$, and then each $X_i$ is drawn from $\operatorname{Bernoulli}(p_i)$; you want to know the distribution of those $p_i$ corresponding to $X_i=1$.

This is equivalent to the final line of my initial answer on a single successful observation: the distribution in question is $\operatorname{Beta}(\alpha+1, \beta)$

To try and convince you, here are a couple of simulations in R of $10^6$ initial draws, first with $\alpha=2, \beta=2$ and then with $\alpha=2, \beta=8$, drawing the simulated densities for the successful cases (black) and comparing this with the theoretical densities for $\operatorname{Beta}(\alpha+1, \beta)$. They are a very close match

set.seed(2021)

positives <- function(samplesize, alpha, beta){
  p <- rbeta(samplesize, alpha, beta)
  X <- rbinom(samplesize, 1, p)
  success <- (X == 1)
  psuccess <- p[success]
  plot(density(psuccess, from=0, to=1))
  curve(dbeta(x, alpha+1, beta), col="red", from=0, to=1, add=TRUE)
  c(mean(success))
  }

positives(10^6, alpha=1, beta=2)

and the chart above, starting with $p\sim \operatorname{Beta}(1,2)$, shows a $\operatorname{Beta}(2,2)$ distribution for the $p_i$ corresponding to $X_i=1$.

enter image description here

and the chart above, starting with $p\sim \operatorname{Beta}(2,2)$, shows a $\operatorname{Beta}(3,2)$ distribution for the $p_i$ corresponding to $X_i=1$. Then

positives(10^6, alpha=2, beta=8)

enter image description here

and the chart above, starting with $p\sim \operatorname{Beta}(2,8)$, shows a $\operatorname{Beta}(3,8)$ distribution for the $p_i$ corresponding to $X_i=1$.

$\endgroup$
5
  • $\begingroup$ Sorry if the question was unclear. I'm not trying to estimate the distribution of $p$ (the parameters $\alpha$ and $\beta$ are already known). I'm trying to estimate the distribution of $p'$ that generated all the successful trials. I updated the question for clarification. $\endgroup$
    – parasu
    May 28 at 22:43
  • $\begingroup$ @parasu You said "$X_i \sim Bernoulli(p)$" which suggested to me that $p$ was the parameter for all the trials $\endgroup$
    – Henry
    May 29 at 0:35
  • $\begingroup$ I understand the confusion now. For every $X_i$ there's a $p_i$ from which $X_i$ is sampled. I updated the post to clarify this. $\endgroup$
    – parasu
    May 29 at 1:19
  • $\begingroup$ @parasu That is in effect the same question worded a different way, but with the same answer. I have now illustrated this $\endgroup$
    – Henry
    May 29 at 1:31
  • $\begingroup$ you're absolutely right. I just checked this as well. It makes sense that it can be reduced to a single draw. Thank you! $\endgroup$
    – parasu
    May 29 at 1:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.