5
$\begingroup$

I know that this question borderlines on Bayesian and frequentist philosophy, somewhat related to this question.

Bayesian point estimation sometimes uses the mean of the posterior distribution. That is, the mean of the distribution of a parameter conditional on the data. True Bayesians would update the posterior when new data become available. However, I am interested in the alternative situation when a new data set (let's say of equal size as a first data set) is used to compute a second posterior mean. If we repeated this process with many new samples of equal size, we get a distribution of posterior means.

Are there results on the form of this distribution? In particular, is this distribution normal (can we apply something like a central limit theorem on the posterior means)?

Improve this question
$\endgroup$
2
  • 1
    $\begingroup$ This might not be exactly what you want, but the Bernstein von Mises thm (en.wikipedia.org/wiki/Bernstein%E2%80%93von_Mises_theorem) gives asymptotic normality of the posterior under certain regularity conditions. $\endgroup$
    – aleshing
    Commented Jun 28, 2018 at 18:39
  • $\begingroup$ @marmle Thanks, yes I know. So is the distribution of means of normal posterior distributions across data sets normal? Probably... and does it hold in general? $\endgroup$
    – tomka
    Commented Jun 28, 2018 at 18:41

1 Answer 1

Reset to default
2
$\begingroup$

Asking for the distribution of the mean after repeated experiments in a Bayesian framework is actually not reasonable, as the Bayesian paradigm will result in a posterior distribution for the mean given the first data which is the most reasonable prior distribution to use when performing inference for the second dataset and so on and so forth. Basically, the resulting distribution of the mean will be the posterior distribution which would have resulted, if you had collected the data as a single large dataset with the original prior. The structure of this posterior distribution is dependent on the structure of the likelihood and to a decreasing amount on the prior function.

The "Bayesian Central Limit Theorem" basically states that under specific circumstances the posterior distribution is approximately a normal distribution. You can read this up here.

Improve this answer
$\endgroup$
3
  • 1
    $\begingroup$ Thanks, this is what I mean by the "true Bayesian" approach. Nevertheless I find my question reasonable to be asked. I could implement a simulation in which I do what I suggest, so the question if there are theoretical results on it seems reasonable. $\endgroup$
    – tomka
    Commented Jun 28, 2018 at 18:03
  • 1
    $\begingroup$ @tomka Most likely estimates derived in the way that you suggest would underlie the classical CLT, as the Bayesian posterior mean estimator is a biased version of the MLE. But I have no work to cite for the assumption. $\endgroup$
    – Alex2006
    Commented Jun 28, 2018 at 21:18
  • 1
    $\begingroup$ I am not sure what the relationship of the MLE and the CLT is? CLT applies to sums of iid. random variables. A MLE is not necessarily a sum of random variables, although it often involves one. $\endgroup$
    – tomka
    Commented Jun 29, 2018 at 9:53

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.