I have a likelihood function as follows:

$$ P(y|x,w, \phi) = \frac{\phi}{2\pi} \exp ^{-0.5 (y-t(x, w)'\phi (y-t(x,w)) } $$

Here $y$ and $x$ are two observed values. $\phi$ is also some given parameter and $t$ is a non-linear transformation parameterised by $w$. $w$ is a two dimensional position vector. $t$ is basically a spatial transformation of $x$. So as usual, the likelihood is the data term and is telling me the probability of observing $y$ for a given $x$, $w$ and $phi$. So the cost function becomes the sum of square difference between $y$ and $t(x, w)$ under the Gaussian noise model where the noise precision is given by $\phi$.

So, I am trying to use Expectation Propagation (EP) to get a Gaussian approximation to the posterior on $w$. Let us call this distribution $q(w)$. EP approximates each of the likelihood terms with factors where each of the factors is a Gaussian. So, in my case it is (ignoring normalisation for the moment):

$$ q(w) = \prod_i f_i(w_i) \approx \prod_i P(y|x,w, \phi) $$

So, the likelihood factorises as a product over the terms and each of these ith terms is approximated by the corresponding factor $f_i(w_i)$. Each of these factor terms is a low dimensional Gaussian on $w_i$. Now, EP is an iterative algorithm where at each iteration we need to do the following computation:

$$ \prod_{i\neq j} f_i(w_i) P(y_i|x_i,w_i, \phi) $$

So, I need to do this multiplication where this factor $f_i$ directly depends on $w_i$ with the exact likelihood i.e. $P(y_i|x_i,w_i, \phi)$. The $f_i$ terms are 2-dimensional Gaussians and so, I need to transform this likelihood so that it is some 2-dimensional distribution over $w_i$ (I realise it will not be a Gaussian, of course).

$t$ is a non-linear transformation that is parameterised using first order spline i.e. simple linear interpolation along the two dimensions. So, t(x, w) can be written as:

$$ t(x, w) = \sum_{d} x_d \; B_{d}(w) $$

Note that $w$ is two dimensional and $B_d$ is the dth first degree B-spline coefficient. So, we are interpolating the observed $x$ at spatial locations $w_d$ using linear interpolation.

  • $\begingroup$ Because you already have "expressed [$P$] as a function of $w$" in your question, please edit it to be more specific about what form you want this expression to have. It would help to explain what form $t$ is already in. $\endgroup$ – whuber Mar 27 '14 at 18:49
  • 2
    $\begingroup$ Sorry it has not been clear. I will have a think about this and edit the question this evening. Thanks you for pointing it out. $\endgroup$ – Luca Mar 27 '14 at 18:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.