Expectation notations In Statistical Decision Theory, one often studies the following two measures (from "The Bayesian Choice"):
Average loss (aka the frequentist risk):
$R\left(\theta,\delta\right) = \mathrm{E}_\theta\left[L\left(\theta,\delta(x)\right)\right] = \int_X L\left(\theta, \delta(x)\right)f(x|\theta)dx$
Posterior expected loss:
$\rho(\pi, d|x) = \mathrm{E}^\pi\left[L\left(\theta,d\right)| x\right] = \int_\Theta L\left(\theta,d\right)\pi\left(\theta|x\right)d\theta$
I am confused by this notation: What do subindices (e.g. $\theta$ in $\mathrm{E}_\theta$), superindices (e.g. $\pi$ in $\mathrm{E}^\pi$) and conditions (e.g.$x$ in $L\left[.|x\right]$ ) represent when defining expectations?
For reference, in the formulas above:


*

*$\delta(x)$ is known as the decision rule (i.e. the allocation of a decision to each outcome x 
$\sim$ $f(x|\theta)$

*The value $\delta(x)$ is also known as the estimate of $\theta$

*The function $\delta$ is known as the estimator

*$\pi$ is the posterior distribution of $\theta$ given $x$

 A: It might be helpful to isolate all the moving parts in a particular expectation. In doing so, recall that an expectation is a Lebesgue-Stieltjes integral. 
I will adapt the notation slightly and use $X$ to denote a random variable, $x$ to denote a fixed value taken by that random variable in its support, that is, $x\in\mathcal{X}\equiv\text{support}(X)$. 


*

*For the average loss, it might be helpful to distinguish between the true parameter $\theta_0$ and the parameter value $\theta$ at which the risk of the estimator $\delta$ is being evaluated.


$$
\begin{align}
R(\theta, \delta) &= \mathbb{E}_{\theta_0} \left(L(\theta, \delta(X)\right)),\, \theta\in \Theta\\
&=\int_\mathcal{X} L(\theta,\delta(x)) \, d\mathbb{P}_{\theta_0}(x)
\end{align}
$$
Note that the integration here is with respect to the true underlying probability measure generating the random variable $X$, which belongs to the model class $\mathcal{P} = \{\mathbb{P}_\theta\mid \theta \in \Theta\}$, $\mathbb{P}_{\theta_0}\in \mathcal{P}$. 
If the probability measure is absolutely continuous with respect to the Lebesgue measure, then we can write it equivalently as you have written it
$$
\begin{align}
R(\theta, \delta) &= \int_\mathcal{X}L(\theta, \delta(x))f(x\mid \theta_0)\, dx
\end{align}
$$


*

*For the posterior loss, which is defined conditional on a value $x \in \mathcal{X}$, that is it is conditional on you having observed the data to be $X=x$, matters are more straightforward. It just says that the expectation is an integral with respect to the posterior which is conditional on the value of the $X$, and if the data were to change, the posterior would change, and so would the posterior expected loss. 


The following notation might clarify matters (or not),
$$
\begin{align}
\rho(\pi(\mid x), d \mid X=x) &= \mathbb{E}^{\pi(\mid x)}\left(L(\theta, d)\mid X=x\right) \\
&= \int_\Theta L(\theta, d)\pi(\theta \mid x)\, d\theta
\end{align}
$$
to indicate exactly where the conditioning variable appears in the integral.
