Let $$y=Hx=3x_1+5x_2$$ I would like to use Bayesian approach to solve the inverse problem, where I am give the above system in addition to an output vector $$y=[12.9,28.9,45.1,61,76.9,92.9,109.1,124.9,141.1,157]$$ (where I synthetically added noise $\sim \mathcal{N}(0,0.1)$).

The posterior solution is:

$$P(x|H,y)=\frac{P(y|x,H)P(x)}{P(H,y)}$$ In this case, let's say we know the prior, $P(x)\sim \mathcal{N}(\mu_p,\sigma_p)$.My problem is how to construct the likelihood. If I assume normally distributed errors, then I can write $$P(y|x,H)= \frac{1}{\sigma_e\sqrt{2\pi}} e^{ -\frac{1}{2}\left(\frac{y-Hx}{\sigma_e}\right)^2 }$$ However, how am I going to get $x$ in the likelihood equation if that is exactly what I am looking for? Reading around I found that I should construct a synthetic $x$ with a range that should more or less cover the expected range of the real $x$. If that was true, then I will be applying the forward model many times, and I might be better off just doing a brute force (generating all possible outputs and see which are same as my output, that would have the corresponding input I am looking for).

Any ideas?

Edit: this problem is artificial. While looking on inverse problems I always found complicated models assumed for the problem and I wanted a simple problem to understand. Here x1 and x2 are vectors and 3 and 5 constants.

  • $\begingroup$ If you are trying to find $P(H|y)$, you aren't looking for $x$, you're looking for $H$, and you have $x$ as data. If you're looking for $x$, then you need a prior on $x$ etc. $\endgroup$
    – jbowman
    Apr 29, 2018 at 1:21
  • $\begingroup$ You still have $P(H|y)$ as your ultimate goal. Do you mean $P(y|H,x)$? $\endgroup$
    – jbowman
    Apr 29, 2018 at 1:47
  • $\begingroup$ I think so. I am just starting so pardon my ignorance. $\endgroup$
    – student1
    Apr 29, 2018 at 1:48
  • $\begingroup$ Yes, thinking through the Bayesian paradigm does take a little work at first. It helps to write out the relationship as you have done under "The posterior solution is:", but you also need to keep track of what is a parameter and what is data. $\endgroup$
    – jbowman
    Apr 29, 2018 at 2:04
  • 1
    $\begingroup$ @Jonas his notation is very confusing because it's nonstandard. It looks to me like $x_1$ and $x_2$ are his parameters and $3$ and $5$ are his data, which are the same for all 10 observations. Note that this appears to be a completely artificial problem, as the OP is adding synthetic noise to get some randomness into the data. $\endgroup$
    – jbowman
    May 5, 2018 at 17:05

1 Answer 1


Assume that the prior on $x= (x_1, x_2)$ is a multivariate normal: $$P(x) = \mathrm{N}(x;\mu_p, \Sigma_p).$$ The likelihood is now correctly given above: The data $y$ is a realisation of the random variable $Hx^\dagger + \eta$, where $\eta \sim \mathrm{N}(0,\sigma^2_e)$ is normal noise and $x^\dagger$ is the true parameter that shall be estimated.

Since the map $x \mapsto Hx$ is linear, and since prior and likelihood are normal, we can compute the posterior analytically. It is given by: $$P(x|H,y) = \mathrm{N}(x;\mu_{\mathrm{post}}, \Sigma_{\mathrm{post}}),$$ where $$\mu_{\mathrm{post}} = E(x|H,y)= \mu_p + \Sigma_pH^T(H\Sigma_pH^T + \sigma^2_e)^{-1}(y-H\mu_p)$$ and $$\Sigma_{\mathrm{post}} = Cov(x|H,y)= \Sigma_p - \Sigma_pH^T(H\Sigma_pH^T + \sigma^2_e)^{-1}H\Sigma_p.$$

These formulae for the posterior mean can be derived with little difficulty - in a sense they are related to the Kalman Filter.

Depending on what your background is, I would suggest you look at one of the following papers/books: Stuart 2010, Allmaras et al. 2013, or Kaipio, Somersalo 2005.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.