# Understanding Randomized Estimators in Statistical Decision Theory

I'm reading through The Bayesian Choice by CP Robert with a particular focus on understanding randomized decision rules vs non-randomized statistical rules in the Bayesian context. In Section 2.3, he introduces a Bayes estimator that seems to have the properties

$$\delta^{\pi}\left(x\right) \equiv argmin_{\delta}\left\{ \int_{\Theta}L\left(\theta,\delta\right)\pi\left(\theta\mid x\right)d\theta\right\}$$

and the value of the integrand is the Bayes risk. My reading is that procedure takes some data, calculates the posterior distribution of the parameters, and determines the decision $$\delta$$ to minimize the loss function.

Later, in Section 2.4, he introduces randomized estimators $$\delta^*$$. The author explains that the action is generated according to a distribution $$\delta^{*}(x,.)$$ in a decision space $$D^{*}$$. We can write the average loss as

$$L\left(\theta,\delta^{*}\left(x\right)\right) = \int_{D}L\left(\theta,a\right)\delta^{*}\left(x,a\right)da$$

One thing I am confused about is the nature of the distribution $$\delta^*(x,.)$$. $$\delta^{*}$$ is supposed to represent the probability that some decision is made given some data $$x$$.

My first question is: does $$\delta^{*}$$ represent all possible decisions or only those optimal in the sense of the definition above for $$\delta^{\pi}$$?

In trying to think about this question, I started by putting the above loss function into the definition of $$\delta^{*}$$ above, and was left with the term

$$\int_{\Theta}\int_{D}L\left(\theta,a\right)\delta^{*}\left(x,a\right)da\pi\left(\theta\mid x\right)d\theta$$

which also shows up in the proof of Theorem 2.4.2 in Robert's book. Re-arranging the integrals, you would get

$$\int_{D}\left\{ \int_{\Theta}L\left(\theta,a\right)\pi\left(\theta\mid x\right)d\theta\right\} \delta^{*}\left(x,a\right)da$$

From above, I imagine I could get the distribution with the following procedure

1. I have a model, some data, and priors
2. I estimate the posterior distribution of the parameters
3. I simulate from the posterior distribution
4. For each set of parameters, I minimize the loss given the parameters to get the optimal decision
5. I am left with the distribution $$\delta^{*}$$ and can calculate the average loss with the above formula.

At this point, I'm curious if the above procedure is what is intended by the above formula.