In section 1.3 of Bickel and Doksum's Mathematical Statistics 2006, the risk function of a nonrandomized rule $d$ is the expectation of loss of the rule wrt the random sample. $$ R(\theta, d) := E_{X\sim P_\theta} l(\theta, d(X)) $$
the risk function of a randomized rule $\delta$ (distributed according to $\lambda$) is further taking expectation wrt the randomized rule. $$ R(\theta, \delta) := E_{\delta \sim \lambda} E_{X\sim P_\theta} l(\theta, d(X)) $$
Q: Are the randomized rule $\delta$ and the sample supposed to be independent? I would like to know if we can exchange the order of the two expectations as: $$ R(\theta, \delta) = E_{X\sim P_\theta} E_{\delta \sim \lambda} l(\theta, \delta(X)) $$
In section 3.3, they studied the relation between minimax rule and Bayes rule in terms of some two-player 0-1 game, where they introduced the nature and a statistician as the two players. The nature chooses a prior distribution $\pi$ over the parameter space $\Theta$, and the statistician chooses a randomized rule. The payoff one player pays to the other is
$$ r(\pi, \delta) := E_{\theta \sim \pi} R(\theta, \delta) = E_{\theta \sim \pi} E_{\delta \sim \lambda} E_{X\sim P_\theta} l(\theta, \delta(X)) $$
Q: Are $\theta$, the randomized rule $\delta$ and the sample supposed to be independent? I would like to know if we can exchange the order of the three expectations as:
$$ r(\pi, \delta) = E_{\delta \sim \lambda} E_{\theta \sim \pi} E_{X\sim P_\theta} l(\theta, \delta(X)) $$
The reason of my question in 2 is:
They said that given $\delta$, $\arg\max_\pi r(\pi, \delta)$, when exists, can be chosen to the point mass probability measure on $\arg\max_\theta R(\theta, \delta)$. It means that, given a randomized rule $\delta$, the least favorite prior distribution $\pi_\delta$ on $\Theta$, can chosen to be a single $\theta_\delta$, i.e. we can remove the randomization of $\theta$
Q: I would like to know if it is also similarly true that given $\pi$, the Bayes randomized rule $\arg\min_\delta r(\pi, \delta)$, when exists, can be chosen to have the point mass probability measure on $\arg\min_d E_{\theta \sim \pi} E_{X\sim P_\theta} l(\theta, d(X)) $?
If yes, it implies a Bayes randomized rule is always a Bayes nonrandomized rule. But I am not sure if this conclusion is correct.
I think that this can be true, if we can exchange the order of $ E_{\delta \sim \lambda}$ and $ E_{\theta \sim \pi}$ in $r(\pi, \delta) $ as in part 2, which is why I ask if $\theta$ and $\delta$ are independent in part 2.
Thanks.
The relevant pages are here for section 1.3 and here for section 3.3.