0
$\begingroup$

This is a problem from chapter 11 of All of Statistics.

The question is as follows: Let $X_1,\ldots,X_n \sim N(\mu,1)$. Let $\theta = e^\mu$. Find the posterior density for $\theta$ analytically and by simulation, where we have $f(\mu) = 1$ the flat prior.

My question is how I should be generating the density by simulation. I have that $\mu | X_1,\ldots,X_n \sim N(\bar{X}, \frac{1}{n})$, which I'm not sure is correct.

Suppose that I do have the density $f_{\mu} = f(\mu | X_1,\ldots,X_n)$. If I were to simulate the density $f_{\theta} = f(\theta | X_1,\ldots,X_n)$, would I just sample repeatedly from $f_{\mu}$ and then apply the exponentiation to it? I feel like I'm missing something here.

Analytically, can I use the delta method here or do I have to apply $f_{\theta} \propto f(x_i|\theta)f(\theta)$ directly?

$\endgroup$
  • $\begingroup$ for the simulation, I think you would need to come up with a procedure to find the posterior and not sample from the analytical result. the analytical solution should be to look a the definition of the posterior definition and i don't think you would need the delta method. $\endgroup$ – asifzuba Aug 16 at 18:22
  • $\begingroup$ What do you mean by "come up with a procedure to find the posterior"? The only thing I can think of that doesn't involve sampling from either $f_{\mu}$ or $f_{\theta}$ (sampling from the latter I feel like would defeat the purpose) is to collect bootstrap samples of the mean $X_1,\ldots,X_n$ and exponentiate them to get a distribution for $e^\mu$. Would that make sense? $\endgroup$ – crossvalidateme Aug 17 at 2:34
  • $\begingroup$ sorry, for being unclear. I've posted an answer below, please let me know if that clarifies the question. thanks! $\endgroup$ – asifzuba Aug 17 at 22:31
  • $\begingroup$ "If I were to simulate the density $f_\theta=f(\theta\,|\,X_1,\,...,\,X_n)$, would I just sample repeatedly from $f_\mu$ and then apply the exponentiation to it?" I think that's all it expects you to do. Then you can plot a histogram of the draws against the density that you computed analytically and see that they agree. $\endgroup$ – jcz Aug 18 at 3:27
  • $\begingroup$ sorry, I feel that that only checks if the rlnorm function is working as advertised and informs nothing to the student about how to think of the bayesian procedure. please see my answer. $\endgroup$ – asifzuba Aug 18 at 18:27
1
$\begingroup$

Analytical:

We know that $\theta = e^\mu$ and $f(\mu) = 1$, so using the idea of change of variable: $$ f(\theta) = f_{\mu}(\ln(\theta))\frac{d}{d\theta}(\ln(\theta)) \\ =\frac{1}{\theta} $$

Now,
$$ f(\theta | X_1, \cdots, X_n) \propto f(X_1, \cdots, X_n | \theta)f(\theta)\\ \propto \frac{\exp( -n(\ln(\theta) - \bar{X})^2)}{\theta} $$

which looks like a log-normal distribution with parameters - $(\bar{X}, 1/n)$

In fact, you can get to this by the definition of log-normal distribution as it is defined as a distribution whose logarithm ($\mu| X_1, \cdots, X_n$ in our case) is normal.

Simulation:

When doing the simulation we bear in mind that we only know the prior distribution & likelihood.

The following code produces the posterior distribution and plots it:

data <- rnorm(100, mean = 5, sd = 1)
grid <- runif(1e4, min = 0.001, max = 250)
prior <- 1/grid
likelihood <- dnorm(mean(data), mean = log(grid), sd = 0.1)
posterior <- likelihood * prior
posterior <- posterior / sum(posterior)
samples <- sample(grid, size = 5e4, replace = T, prob = posterior)
hist(samples, freq = F, ylim = c(0, 0.03), col = "blue", 100, main = "posterior distribution", xlab = "theta")
curve(dlnorm(x, mean  = mean(data), sdlog = 0.1), from = 100, to = 250, add = T, col = "red", lwd = 4)
abline(v = 148, col = "green", lwd = 4, lty = 2)

In words, this code generates points from a 1-dimensional grid over (0, 250] and then computes the posterior density at each sampled point. Finally, it samples the points based on the posterior probability to make the histogram.

Here what that looks like:

enter image description here

The true posterior density of the log-normal is overlayed on top with the red curve. The true value of $\theta$ ($ = e^5 = 148.4$) is shown with a green dashed line.

HTH.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.