# Specifying a Laplace prior using a Gaussian random variable with Gamma variance

I need to place a Laplace prior on a random variable, however, I want to use a Gaussian distribution whose variance is Gamma(1,1) distributed, i.e.,

\begin{align} x &\sim N(\mu,\sigma^2)\\ \sigma^2&\sim\text{Gamma}(1,1) \end{align}

My question is: Given this above setup, what is the corresponding form of my Laplace distribution (location and scale) for this Laplace prior?

• I'm trying to clarify. You have X~normal(0,y) where y~gamma(1,1)? Where does the Laplace come into play? – Eric Sep 4 '14 at 14:24
• Is this code from infer.net? I think what you are doing is modelling that your variable $x$ is a zero mean and you have a gamma prior on the variance. This is usually done because of conjugate properties. I have no idea where the Laplace distribution comes from. I think you need to edit your question to make it more clear. – Luca Sep 4 '14 at 15:48
• Yes exactly this is infer.net. Actually I read this post: social.microsoft.com/Forums/en-US/… I'm trying to have a Laplace prior. but I need to specify the parameters for this Laplace prior. – user3034939 Sep 4 '14 at 15:51
• I am not sure. I know you can generate a Laplace distribution from subtraction of exponential distributions. Perhaps that is what he was hinting at? – Luca Sep 4 '14 at 16:30
• I edited for clarity. So basically if you have a normal random variable whose variance comes from a Gamma(1,1) distribution then that random variable will approximate a Laplace distribution. What the OP is asking is how to determine the parameters of the corresponding Laplace distribution. – Dan Sep 4 '14 at 17:27

## 1 Answer

So the question seems to have stemmed from a discussion on this forum: http://social.microsoft.com/Forums/en-US/47b613d0-177d-4ce3-b54d-2476144ece6b/double-exponential-prior-migrated-from-communityresearchmicrosoftcom?forum=infer.net

The general idea, that the forum alludes to, is that you can build a distribution that resembles a Laplace distribution from a Normal distribution by specifying the variance of the Normal distribution to follow a Gamma distribution, i.e.,

\begin{align} x|\sigma^2 &\sim N(\mu,\sigma^2)\\ \sigma^2&\sim\text{Gamma}(1,1) \end{align}

However, as the OP asks, how to specify the location ($\mu$) and scale ($b$) of the corresponding Laplace distribution is not obvious. And in fact, you can't since the distribution will not be exactly a Laplace distribution, but just something very similar.

To better understand the question we (below) a plot of many Laplace distibutions with differing locations ($\mu$) and scales ($b$).

And so generating random variables from Normal distribution with variance following a Gamma(1,1) distribution will give us something that looks very close to a Laplace distribution.

However, it is only an approximation and is not exact.

On the other hand, there is a way to generate a Laplace distribution (with known location ($\mu$) and scale ($b$)) by using a Normal distribution with variance corresponding to a Rayleigh distribution.

The result is the following:

If $X|Y \sim N(\mu,\sigma=Y)$ with $Y \sim \text{Rayleigh}(b)$ then $X \sim \text{Laplace}(\mu, b)$.

• There is a link between the Rayleigh distribution and the Gamma distribution that may get you close to what you are looking for as well. – Dan Sep 4 '14 at 17:53
• Thanks for your nice explanation :) I think I cannot use this similarity to reach my goal. I should think about that. – user3034939 Sep 5 '14 at 17:00
• I found this paper. Maybe useful for this discussion: Sparse probabilistic projections - by: C´edric Archambeau, Francis R. Bach – user3034939 Oct 3 '14 at 14:42
• This answer is incorrect. Generating random variables from Normal distribution with variance following a Gamma(1,1) distribution gives exactly a Laplace distribution, not an approximation. This follows immediately from your Rayleigh construction since the square of Rayleigh is Gamma with shape 1. – Tom Minka Oct 3 '14 at 15:35