Are these independent: the sample, randomized rule, and random variable having the prior distribution on the parameter space? In section 1.3 of Bickel and Doksum's Mathematical Statistics 2006, 
the risk function of a nonrandomized rule $d$ is the expectation of loss of the rule wrt the random sample.
$$
R(\theta, d) := E_{X\sim P_\theta} l(\theta, d(X))
$$


*

*the risk function of a randomized rule  $\delta$ (distributed
according to $\lambda$) is further taking expectation wrt the
randomized rule. $$ R(\theta, \delta) := E_{\delta \sim \lambda}
E_{X\sim P_\theta} l(\theta, d(X)) $$
Q: Are the randomized rule $\delta$ and the sample supposed to be
independent? I would like to know if we can exchange the order of
the two expectations as:  $$ R(\theta, \delta) = E_{X\sim P_\theta}
E_{\delta \sim \lambda}  l(\theta, \delta(X)) $$

*In section 3.3, they studied the relation between minimax rule and
Bayes rule in terms of some two-player 0-1 game, where they
introduced the nature and a statistician  as the two players. The
nature chooses a prior distribution $\pi$ over the parameter space $\Theta$,
and the statistician chooses a randomized rule. The payoff one
player pays to the other is
$$ r(\pi, \delta) := E_{\theta \sim \pi} R(\theta, \delta) = E_{\theta \sim \pi} E_{\delta \sim \lambda} E_{X\sim
P_\theta} l(\theta, \delta(X)) $$
Q: Are $\theta$, the randomized rule $\delta$ and the sample supposed to be
independent? I would like to know if we can exchange the order of
the three expectations as:  
$$
r(\pi, \delta) = E_{\delta \sim \lambda} E_{\theta \sim \pi} E_{X\sim P_\theta} l(\theta, \delta(X)) 
$$

*The reason of my question in 2 is:
They said that given $\delta$, $\arg\max_\pi r(\pi, \delta)$, when
exists, can be chosen to the point mass probability measure on
$\arg\max_\theta R(\theta, \delta)$. It means that, given a randomized rule $\delta$, the least favorite prior distribution $\pi_\delta$ on $\Theta$, can chosen to be a single $\theta_\delta$, i.e. we can remove the randomization of $\theta$
Q: I would like to know if it is also similarly true that given $\pi$,
the Bayes randomized rule $\arg\min_\delta r(\pi, \delta)$, when exists, can be chosen to have
the point mass probability measure on $\arg\min_d E_{\theta \sim \pi}
E_{X\sim P_\theta} l(\theta, d(X)) $? 
If yes, it implies a Bayes randomized rule is always a Bayes nonrandomized rule. But I am not sure if this conclusion is correct.
I think that this can be true, if
we can exchange the order of $ E_{\delta \sim \lambda}$ and $ E_{\theta
\sim \pi}$ in $r(\pi, \delta) $ as in part 2, which is why I ask if $\theta$ and $\delta$ are independent in part 2.
Thanks.
The relevant pages are here for section 1.3 and here for section 3.3.
 A: 
  
*
  
*Are the randomized rule δ and the sample supposed to be independent?
  

There is no reason to restrict randomised rules in this manner. For instance, randomisation may only occur for some values of the observation/sample.


  
*Are θ, the randomized rule δ and the sample supposed to be independent?
  

In a joint Bayesian model, sample and parameter are dependent since the sampling distribution is the conditional distribution. While the randomised rule cannot depend on the unknown $\theta$ in a functional manner, since it can [functionally] depend on the sample, it possibly does [probabilistically] depend on the unknown $\theta$.


  
*a Bayes randomized rule is always a Bayes nonrandomized rule
  

This is incorrect: A randomised Bayes rule may differ from a non-randomised one and achieve better performances from a risk perspective. For instance, in finite parameter spaces with strictly convex losses, a minimax rule is always a randomised Bayes rule but not necessarily a non-randomised Bayes rule
