I have a nagging question regarding the Normal distribution and maintaining proportionality in Bayesian Inference. Say for example that:

$\pi(\theta|Y) \propto L(Y|\theta)\pi(\theta)$

$Y | \theta \sim N(\theta, \sigma^2)$

$\theta \sim N(\mu, \tau^2)$

If we want to calculate the posterior distribution for $\theta$, we can drop any terms that do not include the parameter of interest (i.e. normalizing constant). This results in the following:

$\pi(\theta|Y) \propto \exp(-\frac{1}{2 \sigma^2}\sum_{i = 1}^{n} (Y_i - \theta)^2) \exp(-\frac{1}{2 \tau^2}(\theta - \mu)^2)$

My question is why is it that we do not drop the $\frac{1}{2}$ in the exponential terms? Why would this no longer be proportional to the distribution? Or do we avoid dropping it for another reason?

Thank you for your help!

  • 1
    $\begingroup$ $e^{ax}=(e^x)^a \ne e^xe^a$ $\endgroup$ – user158565 Dec 2 '18 at 19:04
  • $\begingroup$ Ok but is it not true that $e^{ax} \propto e^x$? As $e^x$ gets larger/ smaller then $e^{ax}$ should also get larger or smaller. @user158565 $\endgroup$ – samvoit4 Dec 2 '18 at 20:01
  • 2
    $\begingroup$ $e^{ax}∝e^x$ means $e^{ax} = ke^x$. So try to find $k$. For $x>0$, $x$ and $x^2$ get larger/ smaller together. But we do not think they are proportional to each other. getting larger/ smaller together and proportional to each other are different stories. $\endgroup$ – user158565 Dec 2 '18 at 20:09

As is pointed out in the comments, exponentials factor via $e^{ab} = (e^a)^b$. Taking proportionality with respect to $\theta$, your posterior kernel is of the form:

$$\begin{equation} \begin{aligned} \pi (\theta|y) &\propto \exp (- \tfrac{1}{2} \cdot f(\theta, y)) \cdot \exp (- \tfrac{1}{2} \cdot g(\theta, y)) \\[6pt] &= \exp (- f(\theta, y))^{1/2} \cdot \exp (- g(\theta, y))^{1/2} \\[6pt] &= \sqrt{\exp (- f(\theta, y)) \cdot \exp (- g(\theta, y))} \\[6pt] &{\propto\kern-8pt \diagup} \exp (- f(\theta, y)) \cdot \exp (- g(\theta, y)). \\[6pt] \end{aligned} \end{equation}$$

Since proportionality does not hold between a function and its square-root, it is not possible to remove the factors $\tfrac{1}{2}$ from the posterior in this case.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.