Forming a prior based on the solution to a linear system On p. 115 of the 4th edition of Machine Learning a Probabilistic Perspective, we have the following:
Let $\epsilon\sim N(0,\frac{1}{\lambda}\text{I})$ and let $L$ be a matrix of dimension $(D-2)\times D$ given by
$$L=\frac{1}{2}\;\begin{bmatrix}
-1 &2  &-1  &  &  & \\ 
 &-1  &2  &-1  &  & \\ 
 &  &\ldots  &  &  & \\ 
 &  &  &-1  &2  &-1 
\end{bmatrix}.$$
Then consider the equation $L{\bf x}=\epsilon$.  According to the book, we can conclude that
$$p({\bf x})= N({\bf x} \;|\; 0,\; (\lambda L^TL)^{-1}).$$
First off, since the system $L{\bf x}=\epsilon$ is underdetermined, there shouldn't be a unique vector of random variables $\bf x$ which solves this system, but instead a whole set of possible solutions, thus I don't see how they determined the distribution was $p({\bf x})$.
Second, it's a little off-putting that they write $(\lambda L^TL)^{-1}$ as the variance, since $L^T L$ is not invertible, even though it works in practice since the density function only uses the precision matrix, the only consequence being that the prior will be improper.  But how should I interpret this 'non-existant' variance, should I pretend it's the pseudo-inverse? 
I still don't see how they came up with $p({\bf x})$ though, could some one explain this to me?  Thanks.
 A: $Lx = \epsilon$ means that $\epsilon$ is a random variable which is a linear combination of a random variable $x$. So here is an inverse problem: given observations of the output $\epsilon$, you want to estimate the input $x$. These are not variables taking fixed values, they are random variables.
Now, how you solve this inverse problem?. Since $\epsilon$ is a Gaussian, so is $x$ (they are related through a linear transformation).
$$
E[Lx] = E[\epsilon] = 0
$$
$$
E[x^{T}L^{T}Lx] = E[\epsilon^{T}\epsilon] = \frac{1}{\lambda}I
$$
This last expression that the variance matrix is $\lambda(L^{T}L)^{-1}$, since that is the transformation that whitens your Gaussian.
As for $L^{T}L$ being singular. This is a case of an improper prior. See this answer where this idea is discussed.
I see that that is a derivative operator (finite difference) so most likely an example of function estimation. The problem here is the null space of your matrix: the constant functions. But as long as you deal with non-trivial functions, you should be able to solve it. And that is the rationale behind using an improper prior in this case.
