I'm reading through The Bayesian Choice by CP Robert with a particular focus on understanding randomized decision rules vs non-randomized statistical rules in the Bayesian context. In Section 2.3, he introduces a Bayes estimator that seems to have the properties

$$ \delta^{\pi}\left(x\right) \equiv argmin_{\delta}\left\{ \int_{\Theta}L\left(\theta,\delta\right)\pi\left(\theta\mid x\right)d\theta\right\} $$

and the value of the integrand is the Bayes risk. My reading is that procedure takes some data, calculates the posterior distribution of the parameters, and determines the decision $\delta$ to minimize the loss function.

Later, in Section 2.4, he introduces randomized estimators $\delta^*$. The author explains that the action is generated according to a distribution $\delta^{*}(x,.)$ in a decision space $D^{*}$. We can write the average loss as

$$ L\left(\theta,\delta^{*}\left(x\right)\right) = \int_{D}L\left(\theta,a\right)\delta^{*}\left(x,a\right)da $$

One thing I am confused about is the nature of the distribution $\delta^*(x,.)$. $\delta^{*}$ is supposed to represent the probability that some decision is made given some data $x$.

My first question is: does $\delta^{*}$ represent all possible decisions or only those optimal in the sense of the definition above for $\delta^{\pi}$?

In trying to think about this question, I started by putting the above loss function into the definition of $\delta^{*}$ above, and was left with the term

$$ \int_{\Theta}\int_{D}L\left(\theta,a\right)\delta^{*}\left(x,a\right)da\pi\left(\theta\mid x\right)d\theta $$

which also shows up in the proof of Theorem 2.4.2 in Robert's book. Re-arranging the integrals, you would get

$$ \int_{D}\left\{ \int_{\Theta}L\left(\theta,a\right)\pi\left(\theta\mid x\right)d\theta\right\} \delta^{*}\left(x,a\right)da $$

From above, I imagine I could get the distribution with the following procedure

  1. I have a model, some data, and priors
  2. I estimate the posterior distribution of the parameters
  3. I simulate from the posterior distribution
  4. For each set of parameters, I minimize the loss given the parameters to get the optimal decision
  5. I am left with the distribution $\delta^{*}$ and can calculate the average loss with the above formula.

At this point, I'm curious if the above procedure is what is intended by the above formula.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.