# Posterior computation for Laplace distribution

I am dealing with being Bayesian and looking for a closed form for a posterior for the scale parameter $\tau$ of a Laplace distribution, such that I can derive a full conditional in my Gibbs sampler.

I do not think I could exploit conjugacy for the Laplace, but any closed-form computation for the posterior would be useful. Any ideas on how to choose the prior and obtain a closed form posterior from which it is known how to sample from?

• To "deal with being a Bayesian," have you considered joining a Bayesian support group? :-) – whuber Apr 30 '18 at 14:16
• The scale parameter should be conjugate with the inverse gamma distribution. – aleshing Apr 30 '18 at 15:46
• @marmle do you have a reference and a derivation for that? – rano Apr 30 '18 at 15:50

The likelihood for $n$ iid observations looks like:

$f(x_1,...x_n|\lambda,\mu) \propto \frac{1}{\lambda^n} exp(-\frac{1}{\lambda}\sum_{i=1}^n|x_i-\mu|)$

Hence a conjugate prior for $\lambda$ with $\mu, x$ known must (thinking only about the algebra) look like:

$f(\lambda) \propto \frac{1}{\lambda^a} exp(-\frac{b}{\lambda})$

As suggested by marmle, this is an Inverse Gamma, although to be nice we'd need to change $a\rightarrow a-1$ and let $a>0, b>0$.

EDIT: To get the updated parameters for $\lambda$:

$f(\lambda|x_1,...x_n, \mu) \propto f(\lambda)f(x_1,...x_n|\lambda,\mu)$

$\propto \frac{1}{\lambda^{a-1}} exp(-\frac{b}{\lambda}) \frac{1}{\lambda^n} exp(-\frac{1}{\lambda}\sum_{i=1}^n|x_i-\mu|)$

$\propto \frac{1}{\lambda^{n+a-1}} exp(-\frac{1}{\lambda}(b+\sum_{i=1}^n|x_i-\mu|))$

• do you have a reference for this and how to update the parameters for the posterior? – rano Apr 30 '18 at 16:23
• No but that's the derivation, so I guess it depends on what you need it for. – conjectures Apr 30 '18 at 17:05
• I need it to sample from efficiently. By looking at the classic pdf of an inverse Gamma, as described in here: en.wikipedia.org/wiki/Inverse-gamma_distribution , do you consider the $\frac{b^{a}}{\Gamma(a)}$ to be the normalizing factor? – rano Apr 30 '18 at 17:29
• Would then be possibile to put a Normal prior on $\mu$ and obtain a NormalInverseGamma conjugate prior, as it is for the classical Normal parameters $\mu$ and $\sigma$? – rano Apr 30 '18 at 17:45
• Sorry, to be clear, you read off the updated params as the expression that is in the same place as the prior, e.g $a$ goes to $n+a$. The location parameter isn't conjugate using the normal distribution. In fact, it is not obvious what it should be (due to absolute value). – conjectures Apr 30 '18 at 18:34

Conditioning on $\lambda$, the posterior on $\mu$ can be expressed in a reasonably closed form: \begin{align} \pi(\mu|x_1,\ldots,x_n) &\propto \pi(\mu) \exp\left(-\frac{1}{\lambda}\sum_{i=1}^n|x_i-\mu|\right)\\ &= \pi(\mu) \exp\left(-\frac{1}{\lambda}\sum_{i=1}^n|x_{(i)}-\mu|\right)\\ &= \pi(\mu) \sum_{j=0}^n \mathbb{I}_{(x_{(j)},x_{(j+1)})}(\mu) \exp\left(-\frac{1}{\lambda}\sum_{i=1}^n|x_{(i)}-\mu|\right)\\ &= \pi(\mu) \sum_{j=0}^n \mathbb{I}_{(x_{(j)},x_{(j+1)})}(\mu) \exp\left(\frac{1}{\lambda}\sum_{i=1}^j[x_{(i)}-\mu|]\right)\exp\left(\frac{1}{\lambda}\sum_{i=j+1}^n[\mu-x_{(i)}|]\right)\\ &=\sum_{j=0}^n \exp\left(\frac{1}{\lambda}\sum_{i=1}^jx_{(i)}-\frac{1}{\lambda}\sum_{i=j+1}^nx_{(i)}\right)\pi(\mu)\exp\left(\frac{2j-n}{\lambda}\mu\right)\mathbb{I}_{(x_{(j)},x_{(j+1)})}(\mu) \end{align} since using a Normal prior $\pi(\mu)$ returns a mixture of truncated Normals. For instance, if the prior is a ${\cal N}(0,\sigma^2)$, then \begin{align} \pi(\mu|x_1,\ldots,x_n) &\propto \sum_{j=0}^n \overbrace{\exp\left(\frac{1}{\lambda}\sum_{i=1}^jx_{(i)}-\frac{1}{\lambda}\sum_{i=j+1}^nx_{(i)}\right)}^{\omega_j}\exp\left(\frac{2j-n}{\lambda}\mu-\frac{\mu^2}{2\sigma^2}\right)\mathbb{I}_{(x_{(j)},x_{(j+1)})}(\mu)\\ &\propto \sum_{j=0}^n \omega_j\exp\left(2\sigma^2\frac{2j-n}{\lambda}\frac{\mu}{2\sigma^2}-\frac{\mu^2}{2\sigma^2}\right)\mathbb{I}_{(x_{(j)},x_{(j+1)})}(\mu)\\ &\propto \sum_{j=0}^n \omega_j\exp\left(\sigma^2\frac{(2j-n)^2}{\lambda^2}\right)\exp\left(\frac{-1}{2\sigma^2}\left[\mu-\frac{(2n-j)\sigma^2}{\lambda}\right]^2\right) \mathbb{I}_{(x_{(j)},x_{(j+1)})}(\mu)\\ \end{align} a mixture of $n+1$ truncated Normal distributions in $\mu$, truncated respectively to the intervals $(x_{(j)},x_{(j+1)})$ with original mean $(2n-j)\sigma^2/\lambda$ and original variance $\sigma^2$. (While obvious, the weights of the mixture are cumbersome in that they imply the coverage probability of $(x_{(j)},x_{(j+1)})$ by the Normal distribution with mean $(2n-j)\sigma^2/\lambda$ and variance $\sigma^2$.)

And the conditional posterior on $\lambda$ associated with an Inverse Gamma ${\cal IG}(a,b)$ is indeed an Inverse Gamma$${\cal IG}\left(a+n,b+\sum_i|x_i-\mu|\right)$$which implies that a two-step Gibbs sampler can be implemented.

• could you please elicit where the mixture of truncated normals lies and how to update its parameters? – rano May 2 '18 at 20:23
• @rano: sorry, I missed this comment of yours. In the final line of the expression of $\pi(\mu|x_1,\ldots)$ I obtained a sum of $n+1$ terms, each involving a Normal density, a weight (independent from $\mu$), and an indicator function on $\mu$. The product of the density with the indicator produces a truncated Normal density, with the weight being corrected accordingly. Since this is the posterior distribution, there is no reason to "update its parameters", unless you are contemplating a sequential update of the posterior, feasible albeit dreary: modify all weights and replace one component. – Xi'an 6 hours ago