# How to calculate the likelihood for Bayesian Inverse Problems

Let $$y=Hx=3x_1+5x_2$$ I would like to use Bayesian approach to solve the inverse problem, where I am give the above system in addition to an output vector $$y=[12.9,28.9,45.1,61,76.9,92.9,109.1,124.9,141.1,157]$$ (where I synthetically added noise $\sim \mathcal{N}(0,0.1)$).

The posterior solution is:

$$P(x|H,y)=\frac{P(y|x,H)P(x)}{P(H,y)}$$ In this case, let's say we know the prior, $P(x)\sim \mathcal{N}(\mu_p,\sigma_p)$.My problem is how to construct the likelihood. If I assume normally distributed errors, then I can write $$P(y|x,H)= \frac{1}{\sigma_e\sqrt{2\pi}} e^{ -\frac{1}{2}\left(\frac{y-Hx}{\sigma_e}\right)^2 }$$ However, how am I going to get $x$ in the likelihood equation if that is exactly what I am looking for? Reading around I found that I should construct a synthetic $x$ with a range that should more or less cover the expected range of the real $x$. If that was true, then I will be applying the forward model many times, and I might be better off just doing a brute force (generating all possible outputs and see which are same as my output, that would have the corresponding input I am looking for).

Any ideas?

Edit: this problem is artificial. While looking on inverse problems I always found complicated models assumed for the problem and I wanted a simple problem to understand. Here x1 and x2 are vectors and 3 and 5 constants.

• If you are trying to find $P(H|y)$, you aren't looking for $x$, you're looking for $H$, and you have $x$ as data. If you're looking for $x$, then you need a prior on $x$ etc. – jbowman Apr 29 '18 at 1:21
• You still have $P(H|y)$ as your ultimate goal. Do you mean $P(y|H,x)$? – jbowman Apr 29 '18 at 1:47
• I think so. I am just starting so pardon my ignorance. – student1 Apr 29 '18 at 1:48
• Yes, thinking through the Bayesian paradigm does take a little work at first. It helps to write out the relationship as you have done under "The posterior solution is:", but you also need to keep track of what is a parameter and what is data. – jbowman Apr 29 '18 at 2:04
• @Jonas his notation is very confusing because it's nonstandard. It looks to me like $x_1$ and $x_2$ are his parameters and $3$ and $5$ are his data, which are the same for all 10 observations. Note that this appears to be a completely artificial problem, as the OP is adding synthetic noise to get some randomness into the data. – jbowman May 5 '18 at 17:05

Assume that the prior on $x= (x_1, x_2)$ is a multivariate normal: $$P(x) = \mathrm{N}(x;\mu_p, \Sigma_p).$$ The likelihood is now correctly given above: The data $y$ is a realisation of the random variable $Hx^\dagger + \eta$, where $\eta \sim \mathrm{N}(0,\sigma^2_e)$ is normal noise and $x^\dagger$ is the true parameter that shall be estimated.
Since the map $x \mapsto Hx$ is linear, and since prior and likelihood are normal, we can compute the posterior analytically. It is given by: $$P(x|H,y) = \mathrm{N}(x;\mu_{\mathrm{post}}, \Sigma_{\mathrm{post}}),$$ where $$\mu_{\mathrm{post}} = E(x|H,y)= \mu_p + \Sigma_pH^T(H\Sigma_pH^T + \sigma^2_e)^{-1}(y-H\mu_p)$$ and $$\Sigma_{\mathrm{post}} = Cov(x|H,y)= \Sigma_p - \Sigma_pH^T(H\Sigma_pH^T + \sigma^2_e)^{-1}H\Sigma_p.$$