# Approximating the expected value of a random variable as a function of the prior mean of a parameter

I have a parameterised statistical model and I'm trying to calculate the expected value of a random variable. I know that the expected value is a function of the value of the unknown parameter. So I have a formula for the expected value as an integral over values of the parameter. But when the integral is complicated, I am looking for an approximation. Let's say I have a prior probability over the parameter values. So I have an expected value for the parameter. How good (or bad) an approximation is it to evaluate the function at the expected value for the parameter?

So I know that $$E[X|\theta] = f(\theta)$$ and thus $$E[X] = \int f(\theta)p(\theta)$$, how good an approximation is $$E[X] \approx f(\overline \theta)$$, where $$\overline \theta$$ is the expected value of $$\theta$$?

My suspicion is that if the function is linear (or close to linear) or the prior on the parameter is peaked about the expected value, then the approximation is reasonable...

• I asked this question on math.stackexchange, but maybe it's better suited to this site? – Seamus Jan 7 '20 at 12:07
• Please remove questions duplicated on different sites. – Xi'an Jan 8 '20 at 18:46

## 2 Answers

If one follows a Bayesian approach, it should apply to all aspects of the model and not only to one particular parameter $$\theta$$. Hence I suggest considering the posterior distribution on $$\mathbb E[X|\theta]$$, which mathematically derives from the posterior distribution on $$\theta$$. Plug-in estimates contradict one of the reasons for adopting a Bayesian viewpoint, namely to convey uncertainty as well as point estimates.

• I understand that what I'm suggesting is not theoretically very nice, but sometimes you want to take shortcuts, and it's helpful to know which shortcuts are somewhat theoretically legit... – Seamus Jan 9 '20 at 12:49

It's priors all the way down.

How good $$\hat{\theta}$$ approximates $$\theta$$ is a conditional probability $$p(\hat{\theta}=\theta|M)$$ where $$M$$ is the model you've used to estimate it in the first place.

A good model approximates well, a bad one doesn't. If you have a spectacularly bad model, e.g. you accidentally flipped the sign on your loss function, your estimated parameters may actually drift further away from the true value the more you train them.