1
$\begingroup$

I have a parameterised statistical model and I'm trying to calculate the expected value of a random variable. I know that the expected value is a function of the value of the unknown parameter. So I have a formula for the expected value as an integral over values of the parameter. But when the integral is complicated, I am looking for an approximation. Let's say I have a prior probability over the parameter values. So I have an expected value for the parameter. How good (or bad) an approximation is it to evaluate the function at the expected value for the parameter?

So I know that $E[X|\theta] = f(\theta)$ and thus $E[X] = \int f(\theta)p(\theta)$, how good an approximation is $E[X] \approx f(\overline \theta)$, where $\overline \theta$ is the expected value of $\theta$?

My suspicion is that if the function is linear (or close to linear) or the prior on the parameter is peaked about the expected value, then the approximation is reasonable...

$\endgroup$
2
  • $\begingroup$ I asked this question on math.stackexchange, but maybe it's better suited to this site? $\endgroup$ – Seamus Jan 7 '20 at 12:07
  • $\begingroup$ Please remove questions duplicated on different sites. $\endgroup$ – Xi'an Jan 8 '20 at 18:46
2
$\begingroup$

If one follows a Bayesian approach, it should apply to all aspects of the model and not only to one particular parameter $\theta$. Hence I suggest considering the posterior distribution on $\mathbb E[X|\theta]$, which mathematically derives from the posterior distribution on $\theta$. Plug-in estimates contradict one of the reasons for adopting a Bayesian viewpoint, namely to convey uncertainty as well as point estimates.

$\endgroup$
1
  • $\begingroup$ I understand that what I'm suggesting is not theoretically very nice, but sometimes you want to take shortcuts, and it's helpful to know which shortcuts are somewhat theoretically legit... $\endgroup$ – Seamus Jan 9 '20 at 12:49
0
$\begingroup$

It's priors all the way down.

How good $\hat{\theta}$ approximates $\theta$ is a conditional probability $p(\hat{\theta}=\theta|M)$ where $M$ is the model you've used to estimate it in the first place.

A good model approximates well, a bad one doesn't. If you have a spectacularly bad model, e.g. you accidentally flipped the sign on your loss function, your estimated parameters may actually drift further away from the true value the more you train them.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.