7
$\begingroup$

Imagine we are performing Bayesian inference with normal-normal conjugate priors. We have some prior:

$$ \mu \sim N(\mu_0, \sigma_0^2). $$

We know we will collect some normally distributed data $x$ with known variance $\sigma^2$ distributed as:

$$ x|\mu \sim N\left(\mu, \sigma^2\right). $$

The prior predictive distribution tells us the distribution of the observation $x$ prior to actually collecting the data:

$$ x \sim N\left(\mu_0, \sigma_0^2 + \sigma^2\right). $$

Once we actually collect a realization of $x$, we will use it to obtain the posterior distribution of $\mu$ using standard Bayesian updating:

$$ \mu~|~x \sim N\left(\frac{\mu_0/\sigma_0^2+x/\sigma^2}{1/\sigma_0^2+1/\sigma^2}, \frac{1}{1/\sigma_0^2+1/\sigma^2}\right). $$

Before actually collecting the data $x$, we know that our posterior variance is going to be $\frac{1}{1/\sigma_0^2+1/\sigma^2}$, but we are unsure what our posterior mean will be, since $x$ is a random variable prior to actually collecting the data. However, from the prior predictive distribution, we do know the distribution of the posterior mean before actually collecting the data $x$:

$$ \frac{\mu_0/\sigma_0^2+x/\sigma^2}{1/\sigma_0^2+1/\sigma^2} \sim N\left(\mu_0, \sigma_0^2 - \frac{1}{1/\sigma_0^2+1/\sigma^2}\right). $$

This leads to my terminology question. What is the name of the distribution of the posterior mean that you have before you actually collect your data? Clearly it's conceptually related to the prior predictive distribution, but I'm interested in the distribution of the posterior mean ($\frac{\mu_0/\sigma_0^2+x/\sigma^2}{1/\sigma_0^2+1/\sigma^2}$ in the example here) instead of the distribution of the observations themselves ($x$ in the example here).


For a bit of motivation of how this sort of question might come up, consider a clinical trial intermediate analysis. We have some data collected thus far, encoded in our prior $\mu\sim N(\mu_0, \sigma_0^2)$. We want to decide whether to run the trial another several months; if we do, we'll collect the additional observation $x$ and obtain some posterior mean. The distribution of the posterior mean before actually collecting the data can inform whether there's much chance of the trial crossing some decision boundary, which can inform whether to continue the trial (and collect $x$) or stop it for futility (and not collect $x$).

$\endgroup$
1
  • 1
    $\begingroup$ I do not think there is any specific name for that distribution, which is a location-scale transform of the prior predictive. $\endgroup$
    – Xi'an
    Commented Jul 19, 2023 at 2:01

1 Answer 1

3
$\begingroup$

What is the name of the distribution of the posterior mean that you have before you actually collect your data?

The name is prior mean. If you don't have any data, the only thing you have are the priors, so there's only the prior distribution. You cannot update the prior if there is no data. There is no intermediate state.

I want to know how far the posterior mean could plausibly move away from the prior mean $\mu_0$ once I collect the data $x$ [...]

I'm afraid that there is no simple answer, it could go almost arbitrarily far. Recall that the posterior is the likelihood times the prior

$$ p(\theta | x) \propto p(x|\theta) \, p(\theta) $$

So the more informative, i.e. stronger the likelihood $p(\theta|x)$ is, the more it influences the result, the same applies to the prior $p(\theta)$, they both impact the result. Now, if your prior is flat $p(\theta) \propto 1$, it has no impact and the likelihood will dominate the result. If it is other "uninformative" prior it would have a limited impact. Now consider another, extreme case where the prior is a degenerate distribution that has all its probability mass centered at $\theta_0$ and is zero everywhere else. In such a case, the posterior could only be a degenerate distribution centered at $\theta_0$ regardless of the data. So the end result would depend on how informative your prior is, and how informative your data is.

If you want to answer the question for a specific model, the easiest approach would be a simulation study where you simulate different plausible values of $x$ and update the prior to see what are the possible posteriors.

$\endgroup$
4
  • 1
    $\begingroup$ Sorry, perhaps I was unclear in the question. I want to know how far the posterior mean could plausibly move away from the prior mean $\mu_0$ once I collect the data $x$ (hence my request about the name for the distribution of the posterior mean). I added an example to the question to motivate this sort of question, which comes up a lot in "expected value of information"-type analyses. Basically, we often want to know if it's worthwhile to invest in collecting data $x$ by knowing how it could potentially impact the posterior mean. $\endgroup$
    – jjj
    Commented Jul 17, 2023 at 21:01
  • $\begingroup$ Are you thinking about Bernstein-von Mises-type convergence? $\endgroup$
    – Durden
    Commented Jul 17, 2023 at 23:45
  • $\begingroup$ Thanks for the update, Tim. Yes, obviously the prior distribution as well as the informativeness of the new data both impact the distribution I'm asking about, as is clear in the example I left in my question (note the dependence on both $\sigma_0$ and $\sigma$). Also, yes clearly we might be able to get the distribution in closed form (as I did in my question) or need to simulate (as you suggest). Unfortunately, your answer don't address my terminology question -- I want to know the name for the distribution of the posterior mean before you actually collect the data $x$. $\endgroup$
    – jjj
    Commented Jul 18, 2023 at 13:25
  • $\begingroup$ @jjj as the answer says, the distribution does not have a name because there's no “intermediate” step. What you are describing is posterior but with imaginary data. As said, you can do a simulation study where you simulate different $x$s, plug into the posterior, and check what could happen. $\endgroup$
    – Tim
    Commented Jul 19, 2023 at 7:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.