1
$\begingroup$

Can the approximating distributions for various factors in Expectation propagation be different distributions but still from the exponential family. For example, I have the following posterior form:

$$ p(w, \lambda, \phi) = p(\phi)p(\lambda)p(w|\lambda) \prod_{i}p(y_{i}|w, \phi, \lambda) $$

So, I need to approximate each of these factors. My question is can, for example, $p(\phi)$ be approximated as a gamma distribution and $p(w|\lambda)$ be a multivariate Gaussian and $p(\lambda)$ be some thing else from the exponential family.

$\endgroup$
1
$\begingroup$

Yes, you can use different exponential families to approximate the marginal for different variables. You only need all messages into a variable to have the same type, so that they can be multiplied together to get the marginal. In your example, $p(w|\lambda)$ (call it factor $a$) can be approximated by the two messages $m_{a \rightarrow w}(w) m_{a \rightarrow \lambda}(\lambda)$ while $p(\lambda)$ (call it factor $b$) is approximated by the single message $m_{b \rightarrow \lambda}(\lambda)$. You need $m_{a \rightarrow \lambda}(\lambda)$ to have the same type as $m_{b \rightarrow \lambda}(\lambda)$ so that they can be multiplied together to get the approximate marginal for $\lambda$.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.