# Expectation notations

In Statistical Decision Theory, one often studies the following two measures (from "The Bayesian Choice"):

Average loss (aka the frequentist risk):

$R\left(\theta,\delta\right) = \mathrm{E}_\theta\left[L\left(\theta,\delta(x)\right)\right] = \int_X L\left(\theta, \delta(x)\right)f(x|\theta)dx$

Posterior expected loss:

$\rho(\pi, d|x) = \mathrm{E}^\pi\left[L\left(\theta,d\right)| x\right] = \int_\Theta L\left(\theta,d\right)\pi\left(\theta|x\right)d\theta$

I am confused by this notation: What do subindices (e.g. $\theta$ in $\mathrm{E}_\theta$), superindices (e.g. $\pi$ in $\mathrm{E}^\pi$) and conditions (e.g.$x$ in $L\left[.|x\right]$ ) represent when defining expectations?

For reference, in the formulas above:

• $\delta(x)$ is known as the decision rule (i.e. the allocation of a decision to each outcome x $\sim$ $f(x|\theta)$
• The value $\delta(x)$ is also known as the estimate of $\theta$
• The function $\delta$ is known as the estimator
• $\pi$ is the posterior distribution of $\theta$ given $x$
-

It might be helpful to isolate all the moving parts in a particular expectation. In doing so, recall that an expectation is a Lebesgue-Stieltjes integral.

I will adapt the notation slightly and use $X$ to denote a random variable, $x$ to denote a fixed value taken by that random variable in its support, that is, $x\in\mathcal{X}\equiv\text{support}(X)$.

• For the average loss, it might be helpful to distinguish between the true parameter $\theta_0$ and the parameter value $\theta$ at which the risk of the estimator $\delta$ is being evaluated.

\begin{align} R(\theta, \delta) &= \mathbb{E}_{\theta_0} \left(L(\theta, \delta(X)\right)),\, \theta\in \Theta\\ &=\int_\mathcal{X} L(\theta,\delta(x)) \, d\mathbb{P}_{\theta_0}(x) \end{align} Note that the integration here is with respect to the true underlying probability measure generating the random variable $X$, which belongs to the model class $\mathcal{P} = \{\mathbb{P}_\theta\mid \theta \in \Theta\}$, $\mathbb{P}_{\theta_0}\in \mathcal{P}$.

If the probability measure is absolutely continuous with respect to the Lebesgue measure, then we can write it equivalently as you have written it \begin{align} R(\theta, \delta) &= \int_\mathcal{X}L(\theta, \delta(x))f(x\mid \theta_0)\, dx \end{align}

• For the posterior loss, which is defined conditional on a value $x \in \mathcal{X}$, that is it is conditional on you having observed the data to be $X=x$, matters are more straightforward. It just says that the expectation is an integral with respect to the posterior which is conditional on the value of the $X$, and if the data were to change, the posterior would change, and so would the posterior expected loss.

The following notation might clarify matters (or not),

\begin{align} \rho(\pi(\mid x), d \mid X=x) &= \mathbb{E}^{\pi(\mid x)}\left(L(\theta, d)\mid X=x\right) \\ &= \int_\Theta L(\theta, d)\pi(\theta \mid x)\, d\theta \end{align} to indicate exactly where the conditioning variable appears in the integral.

-
Thanks! This helps a lot. But why/when do we use sometimes subscripts and other times superscripts with $\mathrm{E}$? What is the convention that authors follow? –  user023472 Oct 30 '12 at 18:22
I do not have a copy of The Bayesian Choice (or any of @xian's other books, which probably reveals me to be a non-Bayesian :)), but I don't think there is a substantive difference. I for one have always used the subscript. –  fg nu Oct 31 '12 at 2:59