Different definitions of Bayes risk

Question

I'm having trouble understanding the proper definition of Bayes risk. Let the data/variate $x \sim P(X|\theta)$, $\theta\in \Theta$, $\pi$ be a distribution on $\Theta$ (prior), $\hat \theta(x)$ be an estimator of $\theta$ based on a variates $x$, and $L$ a loss function between true and estimated parameters $\theta$. It appears to me that there are two different notions of Bayes risk around:

1. For given data , the Bayes risk is defined as $$\mathbb{E}[L(\theta, \hat\theta(x)]$$ with the expectation taken over the prior $\pi$, and $x$ fixed. This is what you'll find in Wikipedia, for instance.
2. In function estimation like wavelet theory, and when contrasting Bayes and minimax estimation, the risk is defined as $$R(\theta, \hat\theta) = \mathbb{E}[L(\theta, \hat\theta(x))]$$ where the expectation is taken over $P(X|\theta)$ (in regression, where $\theta$ is the signal, this means we integrate over the noise). For minimax estimation, we look at the maximum risk $$\max_{\theta\in\Theta} R(\theta, \hat\theta)$$ whereas for the Bayes risk, we put a prior $\pi$ in $\Theta$, so the Bayes risk is defined as $$\mathbb{E}[R(\theta, \hat\theta)]$$ with the expectation taken with respect to $\pi$.

So one is defined for a specific variate, using the loss function, whereas the other is defined for the expected loss (risk) across all variates. I was wondering if these are really two different things, or my understanding is off, or if they are two sides of the same thing. I would appreciate any clarification.

Answer 1

To quote from my book, The Bayesian Choice (2007, Section 2.3, pp. 63-64)

The Bayesian approach to decision theory integrates on the parameter space $\Theta$, since $\theta$ is unknown, instead of integrating on the sampling space ${\cal X}$, as $x$ is observed. It relies on the posterior expected loss \begin{eqnarray*} \rho(\pi,d|x) & = & \mathbb{E}^\pi[L(\theta,d)|x] \\ \tag{1} & = & \int_{\Theta} \mathrm{L}(\theta,d) \pi(\theta|x)\, \text{d}\theta, \end{eqnarray*} which averages the error (i.e., the loss) according to the posterior distribution of the parameter $\theta$, conditional on the observed value $x$. Given $x$, the average error resulting from decision $d$ is actually $\rho(\pi,d|x)$. The posterior expected loss is thus a function of $x$ but this dependence is not an issue, as opposed to the frequentist dependence of the risk on the parameter because $x$, contrary to $\theta$, is known.

Given a prior distribution $\pi$, it is also possible to define the integrated risk, which is the frequentist risk averaged over the values of $\theta$ according to their prior distribution \begin{eqnarray*} r(\pi,\delta) & = & \mathbb{E}^\pi[R(\theta,\delta)] \\ \tag{2} & = & \int_{\Theta} \int_{\cal X} \mathrm{L}(\theta,\delta(x))\, f(x|\theta) \,\text{d}x\ \pi(\theta)\, \text{d}\theta. \end{eqnarray*} One particular appeal of this second concept is that it associates a real number with every estimator, not a function of $\theta$. It therefore induces a total ordering on the set of estimator s, i.e., allows for the direct comparison of estimators. This implies that, while taking into account the prior information through the prior distribution, the Bayesian approach is sufficiently reductive (in a positive sense) to reach an effective decision. Moreover, the above two notions are equivalent in that they lead to the same decision.

A bit further, I use the following definition for the Bayes risk:

A Bayes estimator associated with a prior distribution $\pi$ and a loss function $\mathrm{L}$ is any estimator $\delta^\pi$ which minimizes $r(\pi,\delta)$. For every $x\in\cal{X}$, it is given by $\delta^\pi(x)$, argument of $$\min_d \rho(\pi,d|x)$$ The value $$r(\pi) = r(\pi,\delta^\pi)\tag{3}$$ is then called the Bayes risk.