1
$\begingroup$

I'm reading through The Bayesian Choice by CP Robert with a particular focus on understanding randomized decision rules vs non-randomized statistical rules in the Bayesian context. In Section 2.3, he introduces a Bayes estimator that seems to have the properties

$$ \delta^{\pi}\left(x\right) \equiv argmin_{\delta}\left\{ \int_{\Theta}L\left(\theta,\delta\right)\pi\left(\theta\mid x\right)d\theta\right\} $$

and the value of the integrand is the Bayes risk. My reading is that procedure takes some data, calculates the posterior distribution of the parameters, and determines the decision $\delta$ to minimize the loss function.

Later, in Section 2.4, he introduces randomized estimators $\delta^*$. The author explains that the action is generated according to a distribution $\delta^{*}(x,.)$ in a decision space $D^{*}$. We can write the average loss as

$$ L\left(\theta,\delta^{*}\left(x\right)\right) = \int_{D}L\left(\theta,a\right)\delta^{*}\left(x,a\right)da $$

One thing I am confused about is the nature of the distribution $\delta^*(x,.)$. $\delta^{*}$ is supposed to represent the probability that some decision is made given some data $x$.

My first question is: does $\delta^{*}$ represent all possible decisions or only those optimal in the sense of the definition above for $\delta^{\pi}$?

In trying to think about this question, I started by putting the above loss function into the definition of $\delta^{*}$ above, and was left with the term

$$ \int_{\Theta}\int_{D}L\left(\theta,a\right)\delta^{*}\left(x,a\right)da\pi\left(\theta\mid x\right)d\theta $$

which also shows up in the proof of Theorem 2.4.2 in Robert's book. Re-arranging the integrals, you would get

$$ \int_{D}\left\{ \int_{\Theta}L\left(\theta,a\right)\pi\left(\theta\mid x\right)d\theta\right\} \delta^{*}\left(x,a\right)da $$

From above, I imagine I could get the distribution with the following procedure

  1. I have a model, some data, and priors
  2. I estimate the posterior distribution of the parameters
  3. I simulate from the posterior distribution
  4. For each set of parameters, I minimize the loss given the parameters to get the optimal decision
  5. I am left with the distribution $\delta^{*}$ and can calculate the average loss with the above formula.

At this point, I'm curious if the above procedure is what is intended by the above formula.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.