Understanding Loss functions in Stacked Capsule Autoencoders I was reading Stacked Capsule Autoencoder paper published by Geoff Hinton's group last year in NIPS. While reading section 2.1 about constellation autoencoders I couldn't understand how the expression of parts likelihood is calculated. Likelihood expression given in the paper is as follows
$$
p(x_{1:M}) = \prod_{m=1}^{M} \sum_{k=1}^{K} \sum_{n=1}^{N} \frac{a_k a_{k,n}}{\sum_{i} a_i \sum_{j} a_{i,j}}p(x_m|k,n)
$$
The way I understood it is that it's Gaussian Mixture model. $a_k$ represents the presence probability of $k^{th}$ capsule, $a_{k,n}$ represents the conditional presence probability of $n^{th}$ candidate part {$n^{th}$ Gaussian class} in $k^{th}$ object capsule and $p(x_m | k,n)$  represents the probability that $x_m$ part capsule belongs to $n^{th}$ Gaussian class {$n^{th}$ candidate part} of $k^{th}$ object capsule. So to find the $p(x_1)$ over 'k' object capsules, each having 'n' candidate predictions we marginalize $p(x_1)$ over 'k' and 'n' . So shouldn't the likelihood expression be
$$
 p(x_{1:M}) = \prod_{m=1}^{M} \sum_{k=1}^{K} \sum_{n=1}^{N} a_k a_{k,n} \space  p(x_m|k,n)
$$
From where does $\sum_{i} a_i \sum_{j} a_{i,j}$ is coming in the denominator of expression given in the paper?
 A: The key here is construction of the mixture. The components of the mixture are on the right hand side $a_ka_{kj}p(x_{1:M}|a_k,a_{kj})$. You add them up to get the probability of observing the capsules $p(x_{1:M})$ while having all parts.
On the left hand side you start with the same probability but written as $p(x_{1:m}|\mathrm{parts})p(\mathrm{parts})$, where the probability of all parts present is $p(\mathrm{parts})=\sum_k\sum_ja_ka_{kj}$ and $p(x_m|\mathrm{parts})$ is a constructed mixture. That's why you need to normalize the right hand side by $p(\mathrm{parts})\ne 1$ to get the sought likelihood $p(x_{1:m}|\mathrm{parts})$.
The idea here is not even Bayes theorem, but something like a weighted average.  For instance, let's say are calculating the total weight of bus passengers: $$TW=N(\mathrm{male})*W(\mathrm{male})+N(\mathrm{female})*W(\mathrm{female})$$
where N is the number of passengers and W is the average weight of the group. Then you write the same as $$TW=(N(\mathrm{male})+N(\mathrm{female}))*W(\mathrm{all})$$
This gives the mixture weight $$W(\mathrm{all})=\frac{N(\mathrm{male})*W(\mathrm{male})+N(\mathrm{female})*W(\mathrm{female})}{N(\mathrm{male})+N(\mathrm{female})}$$
It's important to normalize because total num of passengers $N(\mathrm{male})+N(\mathrm{female})$ is not fixed. Changing only num of males impacts the weights of both males and females. In your case adding an object also changes weights of other components of the mixture probability.
