2
$\begingroup$

I need to place a Laplace prior on a random variable, however, I want to use a Gaussian distribution whose variance is Gamma(1,1) distributed, i.e.,

\begin{align} x &\sim N(\mu,\sigma^2)\\ \sigma^2&\sim\text{Gamma}(1,1) \end{align}

My question is: Given this above setup, what is the corresponding form of my Laplace distribution (location and scale) for this Laplace prior?

$\endgroup$
  • $\begingroup$ I'm trying to clarify. You have X~normal(0,y) where y~gamma(1,1)? Where does the Laplace come into play? $\endgroup$ – Eric Sep 4 '14 at 14:24
  • $\begingroup$ Is this code from infer.net? I think what you are doing is modelling that your variable $x$ is a zero mean and you have a gamma prior on the variance. This is usually done because of conjugate properties. I have no idea where the Laplace distribution comes from. I think you need to edit your question to make it more clear. $\endgroup$ – Luca Sep 4 '14 at 15:48
  • $\begingroup$ Yes exactly this is infer.net. Actually I read this post: social.microsoft.com/Forums/en-US/… I'm trying to have a Laplace prior. but I need to specify the parameters for this Laplace prior. $\endgroup$ – user3034939 Sep 4 '14 at 15:51
  • $\begingroup$ I am not sure. I know you can generate a Laplace distribution from subtraction of exponential distributions. Perhaps that is what he was hinting at? $\endgroup$ – Luca Sep 4 '14 at 16:30
  • 1
    $\begingroup$ I edited for clarity. So basically if you have a normal random variable whose variance comes from a Gamma(1,1) distribution then that random variable will approximate a Laplace distribution. What the OP is asking is how to determine the parameters of the corresponding Laplace distribution. $\endgroup$ – Dan Sep 4 '14 at 17:27
2
$\begingroup$

So the question seems to have stemmed from a discussion on this forum: http://social.microsoft.com/Forums/en-US/47b613d0-177d-4ce3-b54d-2476144ece6b/double-exponential-prior-migrated-from-communityresearchmicrosoftcom?forum=infer.net

The general idea, that the forum alludes to, is that you can build a distribution that resembles a Laplace distribution from a Normal distribution by specifying the variance of the Normal distribution to follow a Gamma distribution, i.e.,

\begin{align} x|\sigma^2 &\sim N(\mu,\sigma^2)\\ \sigma^2&\sim\text{Gamma}(1,1) \end{align}

However, as the OP asks, how to specify the location ($\mu$) and scale ($b$) of the corresponding Laplace distribution is not obvious. And in fact, you can't since the distribution will not be exactly a Laplace distribution, but just something very similar.

To better understand the question we (below) a plot of many Laplace distibutions with differing locations ($\mu$) and scales ($b$).

enter image description here

And so generating random variables from Normal distribution with variance following a Gamma(1,1) distribution will give us something that looks very close to a Laplace distribution.

enter image description here

However, it is only an approximation and is not exact.


On the other hand, there is a way to generate a Laplace distribution (with known location ($\mu$) and scale ($b$)) by using a Normal distribution with variance corresponding to a Rayleigh distribution.

The result is the following:

If $X|Y \sim N(\mu,\sigma=Y)$ with $Y \sim \text{Rayleigh}(b)$ then $X \sim \text{Laplace}(\mu, b)$.

$\endgroup$
  • $\begingroup$ There is a link between the Rayleigh distribution and the Gamma distribution that may get you close to what you are looking for as well. $\endgroup$ – Dan Sep 4 '14 at 17:53
  • $\begingroup$ Thanks for your nice explanation :) I think I cannot use this similarity to reach my goal. I should think about that. $\endgroup$ – user3034939 Sep 5 '14 at 17:00
  • $\begingroup$ I found this paper. Maybe useful for this discussion: Sparse probabilistic projections - by: C´edric Archambeau, Francis R. Bach $\endgroup$ – user3034939 Oct 3 '14 at 14:42
  • 1
    $\begingroup$ This answer is incorrect. Generating random variables from Normal distribution with variance following a Gamma(1,1) distribution gives exactly a Laplace distribution, not an approximation. This follows immediately from your Rayleigh construction since the square of Rayleigh is Gamma with shape 1. $\endgroup$ – Tom Minka Oct 3 '14 at 15:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.