5 added 221 characters in body edited Jan 1 '14 at 12:54 Luca 2,0912222 silver badges4242 bronze badges I have a joint probability distribution as given in the figure: In this figure, variables in circles are random variables and variables in squares are constants. So, I can write the joint distribution over the data $$y$$ and the model parameters as: $$P(y, w, \lambda, \phi) = P(y|w, \phi) \times P(w|\lambda) \times P(\lambda) \times P(\phi)$$ Now $$P(w|\lambda)$$ is modelled using a multivariate normal distribution with 0 mean and a covariance matrix scaled by $$\lambda$$. $$P(\lambda)$$ and $$P(\phi)$$ are modelled using gamma distributions. Also, the likelihood term $$P(y|x, w, \phi)$$ is a Gaussian likelihood given by: $$P(y|x, w, \phi) = (\frac{\phi}{2\pi})^{0.5} \exp^{-0.5 e \phi e}$$ The model noise is independent and identically distributed. Now, I am interested in $$P(w, \lambda, \phi|y)$$ which is given by the joint distribution above normalised appropriately by $$P(y)$$ My question is regarding the use of expectation propagation to perform inference on this model? I have been trying to understand EP with little success. Can someone help me understand what approximations I need to make to this model to use EP on it? The prior $$P(w|\lambda)$$ is modelled using a zero mean multivariate normal distribution. $$P(\lambda)$$ is modelled using a Gamma distribution and $$P(\phi)$$ is also a gamma distribution. So, to infer the posterior $$P(w, \lambda, \phi|y)$$, I am confused as to where to start. Should I start with the mean field like approach to assume independence among the posterior distributions of these parameters? $$P(w, \lambda, \phi) \approx q(w) \times q(\lambda) \times q(\phi)$$ I have been struggling with this for weeks now. Any suggestion/reference etc. would be really appreciated. Looking at Minka's lecture notes, I am having issues figuring out whether some factor graph representation for this as that is usually the structure he uses in his examples. I have a joint probability distribution as given in the figure: In this figure, variables in circles are random variables and variables in squares are constants. So, I can write the joint distribution over the data $$y$$ and the model parameters as: $$P(y, w, \lambda, \phi) = P(y|w, \phi) \times P(w|\lambda) \times P(\lambda) \times P(\phi)$$ Now $$P(w|\lambda)$$ is modelled using a multivariate normal distribution with 0 mean and a covariance matrix scaled by $$\lambda$$. $$P(\lambda)$$ and $$P(\phi)$$ are modelled using gamma distributions. Now, I am interested in $$P(w, \lambda, \phi|y)$$ which is given by the joint distribution above normalised appropriately by $$P(y)$$ My question is regarding the use of expectation propagation to perform inference on this model? I have been trying to understand EP with little success. Can someone help me understand what approximations I need to make to this model to use EP on it? The prior $$P(w|\lambda)$$ is modelled using a zero mean multivariate normal distribution. $$P(\lambda)$$ is modelled using a Gamma distribution and $$P(\phi)$$ is also a gamma distribution. So, to infer the posterior $$P(w, \lambda, \phi|y)$$, I am confused as to where to start. Should I start with the mean field like approach to assume independence among the posterior distributions of these parameters? $$P(w, \lambda, \phi) \approx q(w) \times q(\lambda) \times q(\phi)$$ I have been struggling with this for weeks now. Any suggestion/reference etc. would be really appreciated. Looking at Minka's lecture notes, I am having issues figuring out whether some factor graph representation for this as that is usually the structure he uses in his examples. I have a joint probability distribution as given in the figure: In this figure, variables in circles are random variables and variables in squares are constants. So, I can write the joint distribution over the data $$y$$ and the model parameters as: $$P(y, w, \lambda, \phi) = P(y|w, \phi) \times P(w|\lambda) \times P(\lambda) \times P(\phi)$$ Now $$P(w|\lambda)$$ is modelled using a multivariate normal distribution with 0 mean and a covariance matrix scaled by $$\lambda$$. $$P(\lambda)$$ and $$P(\phi)$$ are modelled using gamma distributions. Also, the likelihood term $$P(y|x, w, \phi)$$ is a Gaussian likelihood given by: $$P(y|x, w, \phi) = (\frac{\phi}{2\pi})^{0.5} \exp^{-0.5 e \phi e}$$ The model noise is independent and identically distributed. Now, I am interested in $$P(w, \lambda, \phi|y)$$ which is given by the joint distribution above normalised appropriately by $$P(y)$$ My question is regarding the use of expectation propagation to perform inference on this model? I have been trying to understand EP with little success. Can someone help me understand what approximations I need to make to this model to use EP on it? The prior $$P(w|\lambda)$$ is modelled using a zero mean multivariate normal distribution. $$P(\lambda)$$ is modelled using a Gamma distribution and $$P(\phi)$$ is also a gamma distribution. So, to infer the posterior $$P(w, \lambda, \phi|y)$$, I am confused as to where to start. Should I start with the mean field like approach to assume independence among the posterior distributions of these parameters? $$P(w, \lambda, \phi) \approx q(w) \times q(\lambda) \times q(\phi)$$ I have been struggling with this for weeks now. Any suggestion/reference etc. would be really appreciated. Looking at Minka's lecture notes, I am having issues figuring out whether some factor graph representation for this as that is usually the structure he uses in his examples. 4 Fixed a typo. Made appropriate use of \times in equations edit approved Jan 1 '14 at 11:12 M. Berk 2,08411 gold badge1111 silver badges1717 bronze badges I have a joint probability distribution as given in the figure: In this figure, variables in circles andare random variables and variables in squares are constants. So, I can write the joint distribution over the data $$y$$ and the model parameters as: $$P(y, w, \lambda, \phi) = P(y|w, \phi) * P(w|\lambda) * P(\lambda) * P(\phi)$$$$P(y, w, \lambda, \phi) = P(y|w, \phi) \times P(w|\lambda) \times P(\lambda) \times P(\phi)$$ Now $$P(w|\lambda)$$ is modelled using a multivariate normal distribution with 0 mean and a covariance matrix scaled by $$\lambda$$. $$P(\lambda)$$ and $$P(\phi)$$ are modelled using gamma distributions. Now, I am interested in $$P(w, \lambda, \phi|y)$$ which is given by the joint distribution above normalised appropriately by $$P(y)$$ My question is regarding the use of expectation propagation to perform inference on this model? I have been trying to understand EP with little success. Can someone help me understand what approximations I need to make to this model to use EP on it? The prior $$P(w|\lambda)$$ is modelled using a zero mean multivariate normal distribution. $$P(\lambda)$$ is modelled using a Gamma distribution and $$P(\phi)$$ is also a gamma distribution. So, to infer the posterior $$P(w, \lambda, \phi|y)$$, I am confused as to where to start. Should I start with the mean field like approach to assume independence among the posterior distributions of these parameters? $$P(w, \lambda, \phi) \approx q(w) * q(\lambda) * q(\phi)$$$$P(w, \lambda, \phi) \approx q(w) \times q(\lambda) \times q(\phi)$$ I have been struggling with this for weeks now. Any suggestion/reference etc. would be really appreciated. Looking at Minka's lecture notes, I am having issues figuring out whether some factor graph representation for this as that is usually the structure he uses in his examples. I have a joint probability distribution as given in the figure: In this figure, variables in circles and random variables and variables in squares are constants. So, I can write the joint distribution over the data $$y$$ and the model parameters as: $$P(y, w, \lambda, \phi) = P(y|w, \phi) * P(w|\lambda) * P(\lambda) * P(\phi)$$ Now $$P(w|\lambda)$$ is modelled using a multivariate normal distribution with 0 mean and a covariance matrix scaled by $$\lambda$$. $$P(\lambda)$$ and $$P(\phi)$$ are modelled using gamma distributions. Now, I am interested in $$P(w, \lambda, \phi|y)$$ which is given by the joint distribution above normalised appropriately by $$P(y)$$ My question is regarding the use of expectation propagation to perform inference on this model? I have been trying to understand EP with little success. Can someone help me understand what approximations I need to make to this model to use EP on it? The prior $$P(w|\lambda)$$ is modelled using a zero mean multivariate normal distribution. $$P(\lambda)$$ is modelled using a Gamma distribution and $$P(\phi)$$ is also a gamma distribution. So, to infer the posterior $$P(w, \lambda, \phi|y)$$, I am confused as to where to start. Should I start with the mean field like approach to assume independence among the posterior distributions of these parameters? $$P(w, \lambda, \phi) \approx q(w) * q(\lambda) * q(\phi)$$ I have been struggling with this for weeks now. Any suggestion/reference etc. would be really appreciated. Looking at Minka's lecture notes, I am having issues figuring out whether some factor graph representation for this as that is usually the structure he uses in his examples. I have a joint probability distribution as given in the figure: In this figure, variables in circles are random variables and variables in squares are constants. So, I can write the joint distribution over the data $$y$$ and the model parameters as: $$P(y, w, \lambda, \phi) = P(y|w, \phi) \times P(w|\lambda) \times P(\lambda) \times P(\phi)$$ Now $$P(w|\lambda)$$ is modelled using a multivariate normal distribution with 0 mean and a covariance matrix scaled by $$\lambda$$. $$P(\lambda)$$ and $$P(\phi)$$ are modelled using gamma distributions. Now, I am interested in $$P(w, \lambda, \phi|y)$$ which is given by the joint distribution above normalised appropriately by $$P(y)$$ My question is regarding the use of expectation propagation to perform inference on this model? I have been trying to understand EP with little success. Can someone help me understand what approximations I need to make to this model to use EP on it? The prior $$P(w|\lambda)$$ is modelled using a zero mean multivariate normal distribution. $$P(\lambda)$$ is modelled using a Gamma distribution and $$P(\phi)$$ is also a gamma distribution. So, to infer the posterior $$P(w, \lambda, \phi|y)$$, I am confused as to where to start. Should I start with the mean field like approach to assume independence among the posterior distributions of these parameters? $$P(w, \lambda, \phi) \approx q(w) \times q(\lambda) \times q(\phi)$$ I have been struggling with this for weeks now. Any suggestion/reference etc. would be really appreciated. Looking at Minka's lecture notes, I am having issues figuring out whether some factor graph representation for this as that is usually the structure he uses in his examples. Tweeted twitter.com/#!/StackStats/status/418320521433198592 occurred Jan 1 '14 at 9:59 3 added 172 characters in body edited Jan 1 '14 at 9:13 Luca 2,0912222 silver badges4242 bronze badges I have a joint probability distribution as given in the figure: In this figure, variables in circles and random variables and variables in squares are constants. So, I can write the joint distribution over the data $$y$$ and the model parameters as: $$P(y, w, \lambda, \phi) = P(y|w, \phi) * P(w|\lambda) * P(\lambda) * P(\phi)$$ Now $$P(w|\lambda)$$ is modelled using a multivariate normal distribution with 0 mean and a covariance matrix scaled by $$\lambda$$. $$P(\lambda)$$ and $$P(\phi)$$ are modelled using gamma distributions. Now, I am interested in $$P(w, \lambda, \phi|y)$$ which is given by the joint distribution above normalised appropriately by $$P(y)$$ My question is regarding the use of expectation propagation to perform inference on this model? I have been trying to understand EP with little success. Can someone help me understand what approximations I need to make to this model to use EP on it? The prior $$P(w|\lambda)$$ is modelled using a zero mean multivariate normal distribution. $$P(\lambda)$$ is modelled using a Gamma distribution and $$P(\phi)$$ is also a gamma distribution. So, to infer the posterior $$P(w, \lambda, \phi|y)$$, I am confused as to where to start. Should I start with the mean field like approach to assume independence among the posterior distributions of these parameters? $$P(w, \lambda, \phi) \approx q(w) * q(\lambda) * q(\phi)$$ I have been struggling with this for weeks now. Any suggestion/reference etc. would be really appreciated. Looking at Minka's lecture notes, I am having issues figuring out whether some factor graph representation for this as that is usually the structure he uses in his examples. I have a joint probability distribution as given in the figure: In this figure, variables in circles and random variables and variables in squares are constants. So, I can write the joint distribution over the data $$y$$ and the model parameters as: $$P(y, w, \lambda, \phi) = P(y|w, \phi) * P(w|\lambda) * P(\lambda) * P(\phi)$$ Now $$P(w|\lambda)$$ is modelled using a multivariate normal distribution with 0 mean and a covariance matrix scaled by $$\lambda$$. $$P(\lambda)$$ and $$P(\phi)$$ are modelled using gamma distributions. Now, I am interested in $$P(w, \lambda, \phi|y)$$ which is given by the joint distribution above normalised appropriately by $$P(y)$$ My question is regarding the use of expectation propagation to perform inference on this model? I have been trying to understand EP with little success. Can someone help me understand what approximations I need to make to this model to use EP on it? The prior $$P(w|\lambda)$$ is modelled using a zero mean multivariate normal distribution. $$P(\lambda)$$ is modelled using a Gamma distribution and $$P(\phi)$$ is also a gamma distribution. So, to infer the posterior $$P(w, \lambda, \phi|y)$$, I am confused as to where to start. Should I start with the mean field like approach to assume independence among the posterior distributions of these parameters? $$P(w, \lambda, \phi) \approx q(w) * q(\lambda) * q(\phi)$$ I have been struggling with this for weeks now. Any suggestion/reference etc. would be really appreciated. I have a joint probability distribution as given in the figure: In this figure, variables in circles and random variables and variables in squares are constants. So, I can write the joint distribution over the data $$y$$ and the model parameters as: $$P(y, w, \lambda, \phi) = P(y|w, \phi) * P(w|\lambda) * P(\lambda) * P(\phi)$$ Now $$P(w|\lambda)$$ is modelled using a multivariate normal distribution with 0 mean and a covariance matrix scaled by $$\lambda$$. $$P(\lambda)$$ and $$P(\phi)$$ are modelled using gamma distributions. Now, I am interested in $$P(w, \lambda, \phi|y)$$ which is given by the joint distribution above normalised appropriately by $$P(y)$$ My question is regarding the use of expectation propagation to perform inference on this model? I have been trying to understand EP with little success. Can someone help me understand what approximations I need to make to this model to use EP on it? The prior $$P(w|\lambda)$$ is modelled using a zero mean multivariate normal distribution. $$P(\lambda)$$ is modelled using a Gamma distribution and $$P(\phi)$$ is also a gamma distribution. So, to infer the posterior $$P(w, \lambda, \phi|y)$$, I am confused as to where to start. Should I start with the mean field like approach to assume independence among the posterior distributions of these parameters? $$P(w, \lambda, \phi) \approx q(w) * q(\lambda) * q(\phi)$$ I have been struggling with this for weeks now. Any suggestion/reference etc. would be really appreciated. Looking at Minka's lecture notes, I am having issues figuring out whether some factor graph representation for this as that is usually the structure he uses in his examples. 2 removed signature edited Jan 1 '14 at 8:12 Glen_b♦ 223k2323 gold badges440440 silver badges796796 bronze badges 1 asked Jan 1 '14 at 7:59 Luca 2,0912222 silver badges4242 bronze badges