# Use of expectation propagation for model inference

I have a joint probability distribution as given in the figure:

In this figure, variables in circles are random variables and variables in squares are constants. So, I can write the joint distribution over the data $y$ and the model parameters as:

$$P(y, w, \lambda, \phi) = P(y|w, \phi) \times P(w|\lambda) \times P(\lambda) \times P(\phi)$$

Now $P(w|\lambda)$ is modelled using a multivariate normal distribution with 0 mean and a covariance matrix scaled by $\lambda$. $P(\lambda)$ and $P(\phi)$ are modelled using gamma distributions. Also, the likelihood term $P(y|x, w, \phi)$ is a Gaussian likelihood given by:

$$P(y|x, w, \phi) = (\frac{\phi}{2\pi})^{0.5} \exp^{-0.5 e \phi e}$$

The model noise is independent and identically distributed.

Now, I am interested in $P(w, \lambda, \phi|y)$ which is given by the joint distribution above normalised appropriately by $P(y)$

My question is regarding the use of expectation propagation to perform inference on this model? I have been trying to understand EP with little success.

Can someone help me understand what approximations I need to make to this model to use EP on it? The prior $P(w|\lambda)$ is modelled using a zero mean multivariate normal distribution. $P(\lambda)$ is modelled using a Gamma distribution and $P(\phi)$ is also a gamma distribution. So, to infer the posterior $P(w, \lambda, \phi|y)$, I am confused as to where to start. Should I start with the mean field like approach to assume independence among the posterior distributions of these parameters?

$$P(w, \lambda, \phi) \approx q(w) \times q(\lambda) \times q(\phi)$$

I have been struggling with this for weeks now. Any suggestion/reference etc. would be really appreciated. Looking at Minka's lecture notes, I am having issues figuring out whether some factor graph representation for this as that is usually the structure he uses in his examples.

• Hi Tom, I wonder if you come to this site often or not. But I have one fundamental question about the EP algorithm as described in your papers. When we remove the old estimate of a factor, we are removing it usually from a high dimensional posterior distribution whereas an individual factor itself is usually just specified by a mean and standard deviation. So how does this division operator i.e. $\frac{q(\theta)}{t_i}$ work as $q(\theta)$ is typically high dimensional but the factor $t_i$ is not. – Luca Feb 12 '14 at 0:58