Finding the posterior distribution given an improper prior

Question

Let $X \sim N(\theta, \sigma^2)$ where $\sigma^2$ is known. Let the prior density $\pi(\theta) =1, \theta \in \mathbb{R}$ to be the improper uniform density over the real line. Find the posterior distribution, $\pi(\theta \mid x)$ and posterior mean.

Suppose we denote the sampling distribution as $f(x \mid \theta)$. I know that $$\pi(\theta \mid x) = \frac{f(x \mid \theta)\pi(\theta)}{\int\limits f(x \mid \theta)\pi(\theta)d\theta}.$$ In this particular scenario, it would follow that $\pi(\theta \mid x) = f(x \mid \theta)$. I am confused as to how to get this sampling distribution. In my textbook, there is an example where $n$ iid samples are drawn from a $Bernoulli(p)$ distribution where the prior distribution on $p$ is $beta(\alpha, \beta)$. In that example, they utilized the fact that the sum of the $X_i$'s was $Binomial(n, p)$. However, in my example of an improper prior, even if I take the sum of the data, the distribution would be normal with the mean relying on $\theta$. But in this case, there's nothing I can use about $\theta$. Am I supposed to just use the likelihood function in this case to determine the sampling distribution? If that is the case, can someone help me explain why in some cases we take a sum of the data and other times not?

Answer 1

Actually, $\pi(\theta | x ) = \frac{f(x|\theta)\pi(\theta)}{\int_{\Theta}f(x|\theta)\pi(\theta)d\theta} = \frac{f(x|\theta)1}{\int_{\Theta}f(x|\theta)1d\theta}\propto f(x|\theta)$. I suspect you are uncertain with the idea of a likelihood function.

The likelihood function $f(x|\theta)$ (or $f(\theta;x)$), is a function of the parameter $\theta$. Taking the Binomial example, suppose you have $f(\theta; x=4, n=6)$, then for any $\theta \in [0,1]$, say $\theta=0.3$, $f(\theta)=f(0.3)$ returns you the probability of observing four heads given six flips if your coin was biased with 0.3 probability of heads.Try to plot this for $\theta=0.01, 0.02, \dots, 0.98, 0.99$.

To reiterate, the likelihood function of your parameter. It takes the form of your sampling distribution, except you hold your data constant and take parameter as input instead.

To derive a posterior with improper priors, it is sufficient the integral $\int_\Theta f(\theta;x)\pi(\theta)d\theta < \infty$ (is finite).

See Bayesian Choice - Section 1.5