This might be a naive question, but I still ask it since I couldn't easily find an answer to it.

Suppose we have an unknown parameter, $\theta$, and some function $\theta \mapsto f(\theta)$.

Suppose also that we have an a priori distribution for the value of $\theta$

Finally, we assume for simplicity that we have not obtained any information yet, so our posterior distribution equals the apriori distribution.

So our best estimate of $\theta$ is $\tilde{\theta} = E[\theta]$.

As I understand it, the typical thing to do in Bayesian statistics in order to compute $f$, and please correct me if I'm wrong, is now to simply evalute $f(E[\theta]) = f(\tilde{\theta})$.

However, wouldn't a more correct thing to do be to actually compute $E[f(\theta)]$?

If $f$ is convex for example, it follows that the latter is larger than the former, hence there will be a systematic difference between the two variants.

A reason why I'm asking this is that I have a model where computing $f(\theta)$ for given $\theta$ is expensive, so actually computing $E[f(\theta)]$ is far from desirable.

Edit: Ok, I just read about posterior predictive distributions and it seemed that I was probably wrong in my assumption that one merely used the expected value as the "true" parameter.

  • 4
    $\begingroup$ This is an incorrect assumption: the Bayes estimate of $f(\theta)$ is $\mathbb{E}[f(\theta)]$ $\endgroup$ – Xi'an Mar 10 '16 at 10:11
  • $\begingroup$ The main case when it makes no difference is $\hat{\theta}$ is the median and $f(\theta)$ is a a monotone function. I assume your situation is more complicated, since evaluating some simple function would presumably not be too hard. $\endgroup$ – Björn Mar 10 '16 at 12:00

This is a fairly general problem in parameter estimation. It is usually related to the use of loss functions which do not stay "coherent" under a change of parameters. It affects non-bayesian paparameter estimates as well. For example, if $\hat {\theta} $ is unbiased for $\theta $ then $f (\hat {\theta}) $ is usually not unbiased for $f (\theta) $.

I like to think of picking the right summary for the right space. For example a mean or average is a good summary for symmetrical distributions. And not so good for a skewed distribution. So you should pick the one which has a closer to symmetrical distribution and back transform for the other variable.

Also, this will not make a big difference if your uncertainty/variance is small enough. Just as you get "approximately unbiased" estimates from transforming an unbiased estimate.

Also, do you need parameter estimation or will a histogram or density plot be fine? Much better in some cases because the uncertainty is much clearer.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.