6
$\begingroup$

A uniform prior for a scale parameter (like the variance) is uniform on the logarithmic scale.

What functional form does this prior have on the linear scale? And why so?

$\endgroup$

3 Answers 3

15
$\begingroup$

It's just a standard change of variables; the (monotone & 1-1) transformation is $y = \exp(x)$ with inverse $x=\log(y)$ and Jacobian $\frac{dx}{dy} = \frac{1}{y}$.

With a uniform prior $p_y(y) \propto 1$ on $\mathbb{R}$ we get $p_x(x) = p_y(x(y)) |\frac{dx}{dy}| \propto \frac{1}{y}$ on $(0, \infty)$.

Edit: Wikipedia has a bit on transformations of random variables: http://en.wikipedia.org/wiki/Probability_density_function#Dependent_variables_and_change_of_variables. Similar material will be in any intro probability book. Jim Pitman's "Probability" presents the material in a pretty distinctive way as well IIRC.

$\endgroup$
2
  • 3
    $\begingroup$ just adding to @JMS answer, a great reference to check out about is Bayesian Inference in Statistical Analysis by Box and Tiao. It presents conceptual ideas behind it too. $\endgroup$
    – suncoolsu
    Feb 21, 2011 at 6:20
  • $\begingroup$ Can you explain why we want this transformation? $\endgroup$ Jan 21, 2021 at 18:52
10
$\begingroup$

We are told that the scale parameter is uniform on the logarithmic scale. That means that if x is the scale parameter, then $y=\log(𝑥)$ and the distribution function for $y$ is the one uniform on the logarithmic scale, $p_Y(y) \propto 1$.

Then, applying the Jacobian transformation which comes from the fact that the probability contained in a differential area must be invariant under change of variables, we must have that $p_X(𝑥)=p_Y(y(x))|\frac{dy}{dx}|$. Since $\frac{dy}{dx} \propto \frac{1}{x}$, we obtain $p_X(𝑥) \propto \frac{1}{x}$.

Note: I tried posting this as a comment but I do not have privileges to post comments because I am a new user. The currently accepted answer to the question (given by @JMS) has errors in it. I tried to edit the answer given by @JMS to make the minimum necessary changes but my edit was rejected because people wanted me to put this as a comment or as an answer. Firstly, $p_𝑋(𝑥)$ should end up being a function of $x$, not a function of y. The way @JMS's the answer is phrased right now gives $p_X(x)\propto\frac{1}{y}$. Secondly, there is an error in the Jacobian formulation, it should be $p_X(𝑥)=p_Y(y(x))|\frac{dy}{dx}|$; right now it is given as $p_X(𝑥)=p_Y(x(y))|\frac{dx}{dy}|$. Thirdly, $y=\log(𝑥)$, not $y=\exp(x)$, due to the reason explained in this answer.

$\endgroup$
2
  • $\begingroup$ Why are we told that the scale parameter is uniform on the logarithmic scale? Who decided this? $\endgroup$ Jan 21, 2021 at 18:51
  • $\begingroup$ It was part of the problem the poser of the question was trying to solve: i.e: if a scale parameter is uniform on the logarithmic scale,what functional form does this prior have on the linear scale? $\endgroup$ Jan 21, 2021 at 19:31
2
$\begingroup$

@JMS answer is adequate for the nuts and bolts of changing variables. However, This question may help you a bit with why it is uniform on that scale.

My answer to this question goes through a slightly longer derivation of the "jacobian rule" result given in @JMS's answer. It may help with understanding why the rule applies.

$\endgroup$
4
  • $\begingroup$ +1 for the additional references. My favorite derivation for the change of variables formula starts with the cdf, like in your other answer. $\endgroup$
    – JMS
    Mar 27, 2011 at 15:31
  • $\begingroup$ @JMS - the cdf rule is the only one I don't get confused with, I usually struggle to remember if its $\frac{dy}{dx}$ or $\frac{dx}{dy}$ with the jacobian $\endgroup$ Mar 27, 2011 at 22:40
  • $\begingroup$ same for me - Pitman gives a nice geometric explanation, which is why I referenced it in my answer, but I can't ever remember it when it counts :) When I TA'd a probability class we used this text and some students found it very helpful. $\endgroup$
    – JMS
    Mar 28, 2011 at 1:56
  • $\begingroup$ can you explain why we want this transformation? Why we want the variance invariant on the log scale? $\endgroup$ Jan 21, 2021 at 18:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.