Finding the posterior distribution

Question

Suppose that an observation $X$ is drawn from the following distribution

$f(x|\theta) = \begin{cases} \frac{1}{\theta} & \text{ if } 0 < x < \theta \\ 0 & \text{ if } otherwise \end{cases}$

The prior pdf is

$\xi(\theta) = \begin{cases} \theta e^{-\theta} & \text{ if } \theta > 0 \\ 0 & \text{ if } otherwise \end{cases}$

So I just multiply the likelihood function and the prior to get the posterior

$\xi(\theta|x) \propto \theta e^{-\theta} \frac{1}{\theta}$

$\propto e^{-\theta}$.

Up to this point everything makes sense. Then the solution proceeds to state that

$\xi(\theta|x) = \begin{cases} e^{x - \theta} & \text{ if } \theta > x \\ 0 & \text{ if } otherwise \end{cases}$

Where did the extra $e^x$ in the front come from?

Answer 1

You have the correct form of the posterior distribution, up to a constant of proportionality. Another way of writing what you have is:

$$ \xi(θ|x) = K e^{-\theta}. $$

Since it is a distribution, it must integrate to 1 over the allowable range of $\theta$, which in this case is $x < \theta < \infty$. Think of it as, if you observe a value $x$, you know the value of $\theta$ must be bigger than $x$ and so:

$$ \int_{x}^{\infty} K e^{-\theta} d\theta= 1. $$

After a couple of steps you arrive at $K = e^x$ and so:

$$ \xi(θ|x) = e^x e^{-\theta} = e^{x-\theta}. $$

Answer 2

Two things. Let's start with the fundamental one, which is the reason for your question. The definition of the posterior, which is a consequence of Bayes' Law, is:

$$ P(\theta | x) = \frac{P(x|\theta)P(\theta)}{P(x)}. $$

When our goal is to find the $\theta$ for which $P(\theta|x)$ is maximized, the denominator is irrelevant because it is positive and does not depend on $\theta$. But if we want to explicit calculate the posterior distribution, for instance in the context of Bayesian inference, we can not leave it out.

A second (much smaller) way in which your answer differs from the given answer is that your answer was not piecewise. When we multiply piecewise functions it is important to pay attention to the branches. When you go to compute the marginal distribution $P(x)$ this will be very important as well.