1
$\begingroup$

I have a nagging question regarding the Normal distribution and maintaining proportionality in Bayesian Inference. Say for example that:

$\pi(\theta|Y) \propto L(Y|\theta)\pi(\theta)$

$Y | \theta \sim N(\theta, \sigma^2)$

$\theta \sim N(\mu, \tau^2)$

If we want to calculate the posterior distribution for $\theta$, we can drop any terms that do not include the parameter of interest (i.e. normalizing constant). This results in the following:

$\pi(\theta|Y) \propto \exp(-\frac{1}{2 \sigma^2}\sum_{i = 1}^{n} (Y_i - \theta)^2) \exp(-\frac{1}{2 \tau^2}(\theta - \mu)^2)$

My question is why is it that we do not drop the $\frac{1}{2}$ in the exponential terms? Why would this no longer be proportional to the distribution? Or do we avoid dropping it for another reason?

Thank you for your help!

$\endgroup$
3
  • 1
    $\begingroup$ $e^{ax}=(e^x)^a \ne e^xe^a$ $\endgroup$
    – user158565
    Dec 2, 2018 at 19:04
  • $\begingroup$ Ok but is it not true that $e^{ax} \propto e^x$? As $e^x$ gets larger/ smaller then $e^{ax}$ should also get larger or smaller. @user158565 $\endgroup$
    – samvoit4
    Dec 2, 2018 at 20:01
  • 2
    $\begingroup$ $e^{ax}∝e^x$ means $e^{ax} = ke^x$. So try to find $k$. For $x>0$, $x$ and $x^2$ get larger/ smaller together. But we do not think they are proportional to each other. getting larger/ smaller together and proportional to each other are different stories. $\endgroup$
    – user158565
    Dec 2, 2018 at 20:09

1 Answer 1

4
$\begingroup$

As is pointed out in the comments, exponentials factor via $e^{ab} = (e^a)^b$. Taking proportionality with respect to $\theta$, your posterior kernel is of the form:

$$\begin{equation} \begin{aligned} \pi (\theta|y) &\propto \exp (- \tfrac{1}{2} \cdot f(\theta, y)) \cdot \exp (- \tfrac{1}{2} \cdot g(\theta, y)) \\[6pt] &= \exp (- f(\theta, y))^{1/2} \cdot \exp (- g(\theta, y))^{1/2} \\[6pt] &= \sqrt{\exp (- f(\theta, y)) \cdot \exp (- g(\theta, y))} \\[6pt] &{\propto\kern-8pt \diagup} \exp (- f(\theta, y)) \cdot \exp (- g(\theta, y)). \\[6pt] \end{aligned} \end{equation}$$

Since proportionality does not hold between a function and its square-root, it is not possible to remove the factors $\tfrac{1}{2}$ from the posterior in this case.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.