6
$\begingroup$

In Statistical Decision Theory, one often studies the following two measures (from "The Bayesian Choice"):

Average loss (aka the frequentist risk):

$R\left(\theta,\delta\right) = \mathrm{E}_\theta\left[L\left(\theta,\delta(x)\right)\right] = \int_X L\left(\theta, \delta(x)\right)f(x|\theta)dx$

Posterior expected loss:

$\rho(\pi, d|x) = \mathrm{E}^\pi\left[L\left(\theta,d\right)| x\right] = \int_\Theta L\left(\theta,d\right)\pi\left(\theta|x\right)d\theta$

I am confused by this notation: What do subindices (e.g. $\theta$ in $\mathrm{E}_\theta$), superindices (e.g. $\pi$ in $\mathrm{E}^\pi$) and conditions (e.g.$x$ in $L\left[.|x\right]$ ) represent when defining expectations?

For reference, in the formulas above:

  • $\delta(x)$ is known as the decision rule (i.e. the allocation of a decision to each outcome x $\sim$ $f(x|\theta)$
  • The value $\delta(x)$ is also known as the estimate of $\theta$
  • The function $\delta$ is known as the estimator
  • $\pi$ is the posterior distribution of $\theta$ given $x$
$\endgroup$

1 Answer 1

2
$\begingroup$

It might be helpful to isolate all the moving parts in a particular expectation. In doing so, recall that an expectation is a Lebesgue-Stieltjes integral.

I will adapt the notation slightly and use $X$ to denote a random variable, $x$ to denote a fixed value taken by that random variable in its support, that is, $x\in\mathcal{X}\equiv\text{support}(X)$.

  • For the average loss, it might be helpful to distinguish between the true parameter $\theta_0$ and the parameter value $\theta$ at which the risk of the estimator $\delta$ is being evaluated.

$$ \begin{align} R(\theta, \delta) &= \mathbb{E}_{\theta_0} \left(L(\theta, \delta(X)\right)),\, \theta\in \Theta\\ &=\int_\mathcal{X} L(\theta,\delta(x)) \, d\mathbb{P}_{\theta_0}(x) \end{align} $$ Note that the integration here is with respect to the true underlying probability measure generating the random variable $X$, which belongs to the model class $\mathcal{P} = \{\mathbb{P}_\theta\mid \theta \in \Theta\}$, $\mathbb{P}_{\theta_0}\in \mathcal{P}$.

If the probability measure is absolutely continuous with respect to the Lebesgue measure, then we can write it equivalently as you have written it $$ \begin{align} R(\theta, \delta) &= \int_\mathcal{X}L(\theta, \delta(x))f(x\mid \theta_0)\, dx \end{align} $$

  • For the posterior loss, which is defined conditional on a value $x \in \mathcal{X}$, that is it is conditional on you having observed the data to be $X=x$, matters are more straightforward. It just says that the expectation is an integral with respect to the posterior which is conditional on the value of the $X$, and if the data were to change, the posterior would change, and so would the posterior expected loss.

The following notation might clarify matters (or not),

$$ \begin{align} \rho(\pi(\mid x), d \mid X=x) &= \mathbb{E}^{\pi(\mid x)}\left(L(\theta, d)\mid X=x\right) \\ &= \int_\Theta L(\theta, d)\pi(\theta \mid x)\, d\theta \end{align} $$ to indicate exactly where the conditioning variable appears in the integral.

$\endgroup$
2
  • $\begingroup$ Thanks! This helps a lot. But why/when do we use sometimes subscripts and other times superscripts with $\mathrm{E}$? What is the convention that authors follow? $\endgroup$ Oct 30, 2012 at 18:22
  • $\begingroup$ I do not have a copy of The Bayesian Choice (or any of @xian's other books, which probably reveals me to be a non-Bayesian :)), but I don't think there is a substantive difference. I for one have always used the subscript. $\endgroup$ Oct 31, 2012 at 2:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.