I'm reading through The Bayesian Choice by CP Robert with a particular focus on understanding randomized decision rules vs non-randomized statistical rules in the Bayesian context. In Section 2.3, he introduces a Bayes estimator that seems to have the properties
$$ \delta^{\pi}\left(x\right) \equiv argmin_{\delta}\left\{ \int_{\Theta}L\left(\theta,\delta\right)\pi\left(\theta\mid x\right)d\theta\right\} $$
and the value of the integrand is the Bayes risk. My reading is that procedure takes some data, calculates the posterior distribution of the parameters, and determines the decision $\delta$ to minimize the loss function.
Later, in Section 2.4, he introduces randomized estimators $\delta^*$. The author explains that the action is generated according to a distribution $\delta^{*}(x,.)$ in a decision space $D^{*}$. We can write the average loss as
$$ L\left(\theta,\delta^{*}\left(x\right)\right) = \int_{D}L\left(\theta,a\right)\delta^{*}\left(x,a\right)da $$
One thing I am confused about is the nature of the distribution $\delta^*(x,.)$. $\delta^{*}$ is supposed to represent the probability that some decision is made given some data $x$.
My first question is: does $\delta^{*}$ represent all possible decisions or only those optimal in the sense of the definition above for $\delta^{\pi}$?
In trying to think about this question, I started by putting the above loss function into the definition of $\delta^{*}$ above, and was left with the term
$$ \int_{\Theta}\int_{D}L\left(\theta,a\right)\delta^{*}\left(x,a\right)da\pi\left(\theta\mid x\right)d\theta $$
which also shows up in the proof of Theorem 2.4.2 in Robert's book. Re-arranging the integrals, you would get
$$ \int_{D}\left\{ \int_{\Theta}L\left(\theta,a\right)\pi\left(\theta\mid x\right)d\theta\right\} \delta^{*}\left(x,a\right)da $$
From above, I imagine I could get the distribution with the following procedure
- I have a model, some data, and priors
- I estimate the posterior distribution of the parameters
- I simulate from the posterior distribution
- For each set of parameters, I minimize the loss given the parameters to get the optimal decision
- I am left with the distribution $\delta^{*}$ and can calculate the average loss with the above formula.
At this point, I'm curious if the above procedure is what is intended by the above formula.
