In equation (5) on page 3 on this paper a conditional probability is rewritten using Bayes' formula.

I started using this answer

Can I rewrite conditional probability of three variables like this?

To get

$p(\lambda_1, \lambda_2, n | X_{1:N}) = \frac{p(\lambda_1, \lambda_2, n)}{p(X_{1:N})}$

But I got stuck immediately. Any tips on what they did?

I am not sure what they are taking the proportion with respect to but I thought that if I see the equation fully rewritten with Bayes' formula I could figure it out by myself.


When manipulating probabilities, it is very convenient to know some standard equalities, such as Bayes' theorem. However, it is also very important (perhaps even more so) to work out the dependencies between the involved variables.

The paper is actually simply using Bayes' theorem, but then simplifies the result considering said dependencies (or lack thereof).

A raw application of Bayes' would yield $$ p(\lambda_1, \lambda_2, n|x_{1:N})=\frac{p(x_{1:N}|\lambda_1, \lambda_2, n)p(\lambda_1, \lambda_2, n)}{p(x_{1:N})} \propto p(x_{1:N}|\lambda_1, \lambda_2, n)p(\lambda_1, \lambda_2, n) $$

First, let us examine the dependencies between the parameters. $\lambda_1, \lambda_2$ and $n$. These are drawn independently from a uniform and a Gamma distribution. Therefore, $$ p(\lambda_1, \lambda_2, n) = p(\lambda_1) p(\lambda_2) p(n) $$

Next, let us examine the term $p(x_{1:N}|\lambda_1, \lambda_2, n)$.

First, the $x_i$'s are mutually independent (wether or not we know $\lambda_1, \lambda_2, n$), as they are drawn independently. Therefore $$ p(x_{1:N}|\lambda_1,\lambda_2,n)=\prod_{i=1}^N p(x_i|\lambda_1,\lambda_2,n) $$

The next step is a bit more subtle. The value of $x_i$ depends on either $\lambda_1$ or $\lambda_2$, but not the two of them. Which one of the two influences $x_i$ is determined by $n$, specifically by whether or not $i\leq n$.

So we have $$ p(x_i|\lambda_1,\lambda_2,n)=\begin{cases} p(x_i|\lambda_1) & 1 \leq i \leq n \\ p(x_i|\lambda_2) & n < i \leq N \end{cases} $$

That is, the dependency on $n$ is "encoded" by $i$ and the depedency on either $\lambda_1$ or $\lambda_2$. In other words, $n$ does not influence the value $x_i$ any more than by determining which one of $\lambda_1,\lambda_2$ stays there, so once we choose one of the two, $n$ is no longer relevant.

Hence, $$ p(x_{1:N}|\lambda_1,\lambda_2,n) = \prod_{i=1}^n p(x_i|\lambda_1)\prod_{i=n+1}^N p(x_i|\lambda_2) $$

Finally, the products are collapsed into a joint distribution, presumably for notational simplicity (note that the independencies are employed right afterwards in the paper by expressing the joint in product form). $$ \prod_{i=1}^n p(x_i|\lambda_1)=p(x_{1:n}|\lambda_1) $$ $$ \prod_{i=n+1}^N p(x_i|\lambda_2)=p(x_{n+1:N}|\lambda_2) $$

Putting it all together: $$ p(\lambda_1, \lambda_2, n|X_{1:N}) \propto p(x_{1:n}|\lambda_1) p(x_{n+1:N}|\lambda_2) p(\lambda_1) p(\lambda_2) p(n) $$


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.