2
$\begingroup$

I think I am correctly deriving the binomial-beta conjugate model, but my answer differs slightly from what's on Wikipedia's page on conjugacy.

My solution

Assume that

$$ X_t \sim \text{Binomial}(m, \theta) $$

where $\theta$ is the unknown probability of success and $X_t$ has support over values $0, 1, 2, \dots m$. Let $X = (X_1, X_2, \dots, X_T)$. The likelihood is

$$ p(X \mid \theta) \propto \theta^{\sum x_t} (1 - \theta)^{Tm - \sum x_t}. $$

Place a beta prior on $\theta$ with model hyperpriors $\alpha_0$ and $\beta_0$,

$$ \begin{aligned} p(\theta) &\propto \theta^{\alpha_0 - 1} (1 - \theta)^{\beta_0 - 1}. \end{aligned} $$

Then the posterior is

$$ \begin{aligned} p(\theta \mid X) &\propto \theta^{\alpha_0 + \sum x_t - 1} (1 - \theta)^{\beta_0 + Tm - \sum x_t - 1} \\ &\Downarrow \\ \theta \mid X &\sim \text{Beta}\Big(\alpha_0 + \sum_{t=1}^{T} x_t, \beta_0 + Tm - \sum_{t=1}^{T} x_t \Big). \end{aligned} $$

Wikipedia's solution

Wikipedia writes

$$ \theta \mid X \sim \text{Beta}\Big(\alpha_0 + \sum_{t=1}^{T} x_t, \beta_0 + \sum_{t=1}^{T} N_t - \sum_{t=1}^{T} x_t \Big) $$

In other words, they write $\sum_{t=1}^{T} N_t$ where I write $Tm$. Who is correct? What is $N_t$?

$\endgroup$

1 Answer 1

2
$\begingroup$

Binomial distribution is a distribution for the total number of successes in $n$ independent Bernoulli trials, each with probability of success $\theta$. It doesn't really matter in how many experiments did you observe the results, you care only about total number of successes in total number of trials. You could as well shuffle the data and it would not change the results because the trials are i.i.d., same applies to re-groupping and aggregating it.

$\endgroup$
3
  • 1
    $\begingroup$ If it helps, one explanation for the above is that the total number of successes is sufficient in the classical sense of sufficient statistics. $\endgroup$
    – Xi'an
    Jan 29, 2020 at 18:34
  • $\begingroup$ Where is the mistake in my derivation? I don't see how what you're saying—which I believe I already understand—maps onto my formalism vs. Wikipedia's. For example, I don't know what $N_t$ is. $\endgroup$
    – jds
    Jan 29, 2020 at 19:08
  • 1
    $\begingroup$ @gwg $Tm = \sum_{t=1}^T N_t$, $N_t$ are counts in individual experiments. You just assumed $N_1 = N_2 = \dots = N_T = m$. $\endgroup$
    – Tim
    Jan 29, 2020 at 19:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.