In 'Bayesian Data Analysis' (Gelman, Carlin, Stern and Rubin) on page 64 it reads:

"If the density of $y$ is such that $p(y-\theta|\theta)$ is a function that is free of $\theta$ and $y$, say $f(u)$ where $u = y - \theta$, then $y - \theta$ is a pivotal quantity, and $\theta$ is called a pure location parameter. In such a case, it is reasonable that a noninformative prior distribution for $\theta$ would give $f(y - \theta)$ for the posterior ditsribution, $p(y - \theta|y)$. That is, under the posterior distribution, $y-\theta$ should still be a pivotal quantity, whose distribution is free of both $\theta$ and $y$. Under this condition, using Bayes' rule, $p(y - \theta|y) \propto p(\theta)p(y - \theta|\theta)$..."

Maybe I'm being dense, but shouldn't Bayes' rule say something like $p(y - \theta|y) \propto p(y - \theta)p(y|y-\theta)$? What am I forgetting to remember here?

  • $\begingroup$ $p(y - \theta|y) = p(y - \theta,y)/p(y) \propto p(y - \theta,y) \propto p(y , \theta) \propto p(y - \theta,\theta)\propto p(\theta)p(y - \theta|\theta)$ -- unless I did something silly there, I guess it would go something like that. $\endgroup$ – Glen_b -Reinstate Monica Oct 3 '14 at 3:18
  • $\begingroup$ I guess I'm confused as to why $p(y-\theta, y) \propto p(y,\theta)$. I see how the Jacobian is one, but I don't see how that's the same thing. I'm running through some examples in my head...like if $y,\theta$ were two independent standard normals, and when you transformed them it would correlate them. Off the top of my head I can't see how that's proportional in two variables. $\endgroup$ – Taylor Oct 3 '14 at 17:17
  • $\begingroup$ Since the Jacobian is 1 the intuition should be a bit easier to generate. Think about being in a small region near $(y,\theta)$, say of size $dy\, d\theta$. Whether you define where you are in terms of $(y-\theta,y)$ or $(y,\theta)$ or $(y-\theta,\theta)$, the probability you're in that little region will be the same. $\endgroup$ – Glen_b -Reinstate Monica Oct 4 '14 at 1:39
  • $\begingroup$ I guess it's just notational confusion. Usually when I think of change of variables, I rename the new stuff. If you don't rename things, and you don't have to worry about a Jacobian, it's exactly the same density when you write it down. I'm not crazy, though right? Writing the same density down as a density for something else. A bit sloppy I'd say. But thanks though @Glen_b $\endgroup$ – Taylor Oct 6 '14 at 22:45
  • 1
    $\begingroup$ I see, thanks - their notation is pretty "compact", I would say :-). $\endgroup$ – Christoph Hanck Jan 14 '16 at 13:49

They're both true. To justify the way that the book mentions, it might help to use slightly different notation.

If $\theta$ is a location parameter, then set $U = Y-\theta$ and observe that $$ p_{U \mid Y}(u \mid y) \propto p_{U,Y}(u,y) = p_{U,\theta}(u, \theta)|1| = p_{U \mid\theta}(u \mid \theta) p_{\theta}(\theta). $$ We can use the $\propto$ sign because the normalizing constant is free of $\theta$. The first equality is by the transformation theorem, where we are transforming $(\theta,u) \mapsto (y,u)$. The last property is just the definition of a conditional density.

Stumbling on the same paragraph more than $4$ years later...

| cite | improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.