5 added 221 characters in body
source | link

I have a joint probability distribution as given in the figure:

enter image description here

In this figure, variables in circles are random variables and variables in squares are constants. So, I can write the joint distribution over the data $y$ and the model parameters as:

$$ P(y, w, \lambda, \phi) = P(y|w, \phi) \times P(w|\lambda) \times P(\lambda) \times P(\phi) $$

Now $P(w|\lambda)$ is modelled using a multivariate normal distribution with 0 mean and a covariance matrix scaled by $\lambda$. $P(\lambda)$ and $P(\phi)$ are modelled using gamma distributions. Also, the likelihood term $P(y|x, w, \phi)$ is a Gaussian likelihood given by:

$$ P(y|x, w, \phi) = (\frac{\phi}{2\pi})^{0.5} \exp^{-0.5 e \phi e} $$

The model noise is independent and identically distributed.

Now, I am interested in $P(w, \lambda, \phi|y)$ which is given by the joint distribution above normalised appropriately by $P(y)$

My question is regarding the use of expectation propagation to perform inference on this model? I have been trying to understand EP with little success.

Can someone help me understand what approximations I need to make to this model to use EP on it? The prior $P(w|\lambda)$ is modelled using a zero mean multivariate normal distribution. $P(\lambda)$ is modelled using a Gamma distribution and $P(\phi)$ is also a gamma distribution. So, to infer the posterior $P(w, \lambda, \phi|y)$, I am confused as to where to start. Should I start with the mean field like approach to assume independence among the posterior distributions of these parameters?

$$ P(w, \lambda, \phi) \approx q(w) \times q(\lambda) \times q(\phi) $$

I have been struggling with this for weeks now. Any suggestion/reference etc. would be really appreciated. Looking at Minka's lecture notes, I am having issues figuring out whether some factor graph representation for this as that is usually the structure he uses in his examples.

I have a joint probability distribution as given in the figure:

enter image description here

In this figure, variables in circles are random variables and variables in squares are constants. So, I can write the joint distribution over the data $y$ and the model parameters as:

$$ P(y, w, \lambda, \phi) = P(y|w, \phi) \times P(w|\lambda) \times P(\lambda) \times P(\phi) $$

Now $P(w|\lambda)$ is modelled using a multivariate normal distribution with 0 mean and a covariance matrix scaled by $\lambda$. $P(\lambda)$ and $P(\phi)$ are modelled using gamma distributions.

Now, I am interested in $P(w, \lambda, \phi|y)$ which is given by the joint distribution above normalised appropriately by $P(y)$

My question is regarding the use of expectation propagation to perform inference on this model? I have been trying to understand EP with little success.

Can someone help me understand what approximations I need to make to this model to use EP on it? The prior $P(w|\lambda)$ is modelled using a zero mean multivariate normal distribution. $P(\lambda)$ is modelled using a Gamma distribution and $P(\phi)$ is also a gamma distribution. So, to infer the posterior $P(w, \lambda, \phi|y)$, I am confused as to where to start. Should I start with the mean field like approach to assume independence among the posterior distributions of these parameters?

$$ P(w, \lambda, \phi) \approx q(w) \times q(\lambda) \times q(\phi) $$

I have been struggling with this for weeks now. Any suggestion/reference etc. would be really appreciated. Looking at Minka's lecture notes, I am having issues figuring out whether some factor graph representation for this as that is usually the structure he uses in his examples.

I have a joint probability distribution as given in the figure:

enter image description here

In this figure, variables in circles are random variables and variables in squares are constants. So, I can write the joint distribution over the data $y$ and the model parameters as:

$$ P(y, w, \lambda, \phi) = P(y|w, \phi) \times P(w|\lambda) \times P(\lambda) \times P(\phi) $$

Now $P(w|\lambda)$ is modelled using a multivariate normal distribution with 0 mean and a covariance matrix scaled by $\lambda$. $P(\lambda)$ and $P(\phi)$ are modelled using gamma distributions. Also, the likelihood term $P(y|x, w, \phi)$ is a Gaussian likelihood given by:

$$ P(y|x, w, \phi) = (\frac{\phi}{2\pi})^{0.5} \exp^{-0.5 e \phi e} $$

The model noise is independent and identically distributed.

Now, I am interested in $P(w, \lambda, \phi|y)$ which is given by the joint distribution above normalised appropriately by $P(y)$

My question is regarding the use of expectation propagation to perform inference on this model? I have been trying to understand EP with little success.

Can someone help me understand what approximations I need to make to this model to use EP on it? The prior $P(w|\lambda)$ is modelled using a zero mean multivariate normal distribution. $P(\lambda)$ is modelled using a Gamma distribution and $P(\phi)$ is also a gamma distribution. So, to infer the posterior $P(w, \lambda, \phi|y)$, I am confused as to where to start. Should I start with the mean field like approach to assume independence among the posterior distributions of these parameters?

$$ P(w, \lambda, \phi) \approx q(w) \times q(\lambda) \times q(\phi) $$

I have been struggling with this for weeks now. Any suggestion/reference etc. would be really appreciated. Looking at Minka's lecture notes, I am having issues figuring out whether some factor graph representation for this as that is usually the structure he uses in his examples.

4 Fixed a typo. Made appropriate use of \times in equations
source | link

I have a joint probability distribution as given in the figure:

enter image description here

In this figure, variables in circles andare random variables and variables in squares are constants. So, I can write the joint distribution over the data $y$ and the model parameters as:

$$ P(y, w, \lambda, \phi) = P(y|w, \phi) * P(w|\lambda) * P(\lambda) * P(\phi) $$$$ P(y, w, \lambda, \phi) = P(y|w, \phi) \times P(w|\lambda) \times P(\lambda) \times P(\phi) $$

Now $P(w|\lambda)$ is modelled using a multivariate normal distribution with 0 mean and a covariance matrix scaled by $\lambda$. $P(\lambda)$ and $P(\phi)$ are modelled using gamma distributions.

Now, I am interested in $P(w, \lambda, \phi|y)$ which is given by the joint distribution above normalised appropriately by $P(y)$

My question is regarding the use of expectation propagation to perform inference on this model? I have been trying to understand EP with little success.

Can someone help me understand what approximations I need to make to this model to use EP on it? The prior $P(w|\lambda)$ is modelled using a zero mean multivariate normal distribution. $P(\lambda)$ is modelled using a Gamma distribution and $P(\phi)$ is also a gamma distribution. So, to infer the posterior $P(w, \lambda, \phi|y)$, I am confused as to where to start. Should I start with the mean field like approach to assume independence among the posterior distributions of these parameters?

$$ P(w, \lambda, \phi) \approx q(w) * q(\lambda) * q(\phi) $$$$ P(w, \lambda, \phi) \approx q(w) \times q(\lambda) \times q(\phi) $$

I have been struggling with this for weeks now. Any suggestion/reference etc. would be really appreciated. Looking at Minka's lecture notes, I am having issues figuring out whether some factor graph representation for this as that is usually the structure he uses in his examples.

I have a joint probability distribution as given in the figure:

enter image description here

In this figure, variables in circles and random variables and variables in squares are constants. So, I can write the joint distribution over the data $y$ and the model parameters as:

$$ P(y, w, \lambda, \phi) = P(y|w, \phi) * P(w|\lambda) * P(\lambda) * P(\phi) $$

Now $P(w|\lambda)$ is modelled using a multivariate normal distribution with 0 mean and a covariance matrix scaled by $\lambda$. $P(\lambda)$ and $P(\phi)$ are modelled using gamma distributions.

Now, I am interested in $P(w, \lambda, \phi|y)$ which is given by the joint distribution above normalised appropriately by $P(y)$

My question is regarding the use of expectation propagation to perform inference on this model? I have been trying to understand EP with little success.

Can someone help me understand what approximations I need to make to this model to use EP on it? The prior $P(w|\lambda)$ is modelled using a zero mean multivariate normal distribution. $P(\lambda)$ is modelled using a Gamma distribution and $P(\phi)$ is also a gamma distribution. So, to infer the posterior $P(w, \lambda, \phi|y)$, I am confused as to where to start. Should I start with the mean field like approach to assume independence among the posterior distributions of these parameters?

$$ P(w, \lambda, \phi) \approx q(w) * q(\lambda) * q(\phi) $$

I have been struggling with this for weeks now. Any suggestion/reference etc. would be really appreciated. Looking at Minka's lecture notes, I am having issues figuring out whether some factor graph representation for this as that is usually the structure he uses in his examples.

I have a joint probability distribution as given in the figure:

enter image description here

In this figure, variables in circles are random variables and variables in squares are constants. So, I can write the joint distribution over the data $y$ and the model parameters as:

$$ P(y, w, \lambda, \phi) = P(y|w, \phi) \times P(w|\lambda) \times P(\lambda) \times P(\phi) $$

Now $P(w|\lambda)$ is modelled using a multivariate normal distribution with 0 mean and a covariance matrix scaled by $\lambda$. $P(\lambda)$ and $P(\phi)$ are modelled using gamma distributions.

Now, I am interested in $P(w, \lambda, \phi|y)$ which is given by the joint distribution above normalised appropriately by $P(y)$

My question is regarding the use of expectation propagation to perform inference on this model? I have been trying to understand EP with little success.

Can someone help me understand what approximations I need to make to this model to use EP on it? The prior $P(w|\lambda)$ is modelled using a zero mean multivariate normal distribution. $P(\lambda)$ is modelled using a Gamma distribution and $P(\phi)$ is also a gamma distribution. So, to infer the posterior $P(w, \lambda, \phi|y)$, I am confused as to where to start. Should I start with the mean field like approach to assume independence among the posterior distributions of these parameters?

$$ P(w, \lambda, \phi) \approx q(w) \times q(\lambda) \times q(\phi) $$

I have been struggling with this for weeks now. Any suggestion/reference etc. would be really appreciated. Looking at Minka's lecture notes, I am having issues figuring out whether some factor graph representation for this as that is usually the structure he uses in his examples.

    Tweeted twitter.com/#!/StackStats/status/418320521433198592
3 added 172 characters in body
source | link

I have a joint probability distribution as given in the figure:

enter image description here

In this figure, variables in circles and random variables and variables in squares are constants. So, I can write the joint distribution over the data $y$ and the model parameters as:

$$ P(y, w, \lambda, \phi) = P(y|w, \phi) * P(w|\lambda) * P(\lambda) * P(\phi) $$

Now $P(w|\lambda)$ is modelled using a multivariate normal distribution with 0 mean and a covariance matrix scaled by $\lambda$. $P(\lambda)$ and $P(\phi)$ are modelled using gamma distributions.

Now, I am interested in $P(w, \lambda, \phi|y)$ which is given by the joint distribution above normalised appropriately by $P(y)$

My question is regarding the use of expectation propagation to perform inference on this model? I have been trying to understand EP with little success.

Can someone help me understand what approximations I need to make to this model to use EP on it? The prior $P(w|\lambda)$ is modelled using a zero mean multivariate normal distribution. $P(\lambda)$ is modelled using a Gamma distribution and $P(\phi)$ is also a gamma distribution. So, to infer the posterior $P(w, \lambda, \phi|y)$, I am confused as to where to start. Should I start with the mean field like approach to assume independence among the posterior distributions of these parameters?

$$ P(w, \lambda, \phi) \approx q(w) * q(\lambda) * q(\phi) $$

I have been struggling with this for weeks now. Any suggestion/reference etc. would be really appreciated. Looking at Minka's lecture notes, I am having issues figuring out whether some factor graph representation for this as that is usually the structure he uses in his examples.

I have a joint probability distribution as given in the figure:

enter image description here

In this figure, variables in circles and random variables and variables in squares are constants. So, I can write the joint distribution over the data $y$ and the model parameters as:

$$ P(y, w, \lambda, \phi) = P(y|w, \phi) * P(w|\lambda) * P(\lambda) * P(\phi) $$

Now $P(w|\lambda)$ is modelled using a multivariate normal distribution with 0 mean and a covariance matrix scaled by $\lambda$. $P(\lambda)$ and $P(\phi)$ are modelled using gamma distributions.

Now, I am interested in $P(w, \lambda, \phi|y)$ which is given by the joint distribution above normalised appropriately by $P(y)$

My question is regarding the use of expectation propagation to perform inference on this model? I have been trying to understand EP with little success.

Can someone help me understand what approximations I need to make to this model to use EP on it? The prior $P(w|\lambda)$ is modelled using a zero mean multivariate normal distribution. $P(\lambda)$ is modelled using a Gamma distribution and $P(\phi)$ is also a gamma distribution. So, to infer the posterior $P(w, \lambda, \phi|y)$, I am confused as to where to start. Should I start with the mean field like approach to assume independence among the posterior distributions of these parameters?

$$ P(w, \lambda, \phi) \approx q(w) * q(\lambda) * q(\phi) $$

I have been struggling with this for weeks now. Any suggestion/reference etc. would be really appreciated.

I have a joint probability distribution as given in the figure:

enter image description here

In this figure, variables in circles and random variables and variables in squares are constants. So, I can write the joint distribution over the data $y$ and the model parameters as:

$$ P(y, w, \lambda, \phi) = P(y|w, \phi) * P(w|\lambda) * P(\lambda) * P(\phi) $$

Now $P(w|\lambda)$ is modelled using a multivariate normal distribution with 0 mean and a covariance matrix scaled by $\lambda$. $P(\lambda)$ and $P(\phi)$ are modelled using gamma distributions.

Now, I am interested in $P(w, \lambda, \phi|y)$ which is given by the joint distribution above normalised appropriately by $P(y)$

My question is regarding the use of expectation propagation to perform inference on this model? I have been trying to understand EP with little success.

Can someone help me understand what approximations I need to make to this model to use EP on it? The prior $P(w|\lambda)$ is modelled using a zero mean multivariate normal distribution. $P(\lambda)$ is modelled using a Gamma distribution and $P(\phi)$ is also a gamma distribution. So, to infer the posterior $P(w, \lambda, \phi|y)$, I am confused as to where to start. Should I start with the mean field like approach to assume independence among the posterior distributions of these parameters?

$$ P(w, \lambda, \phi) \approx q(w) * q(\lambda) * q(\phi) $$

I have been struggling with this for weeks now. Any suggestion/reference etc. would be really appreciated. Looking at Minka's lecture notes, I am having issues figuring out whether some factor graph representation for this as that is usually the structure he uses in his examples.

2 removed signature
source | link
1
source | link