4
$\begingroup$

In section 1.3 of Bickel and Doksum's Mathematical Statistics 2006, the risk function of a nonrandomized rule $d$ is the expectation of loss of the rule wrt the random sample. $$ R(\theta, d) := E_{X\sim P_\theta} l(\theta, d(X)) $$

  1. the risk function of a randomized rule $\delta$ (distributed according to $\lambda$) is further taking expectation wrt the randomized rule. $$ R(\theta, \delta) := E_{\delta \sim \lambda} E_{X\sim P_\theta} l(\theta, d(X)) $$

    Q: Are the randomized rule $\delta$ and the sample supposed to be independent? I would like to know if we can exchange the order of the two expectations as: $$ R(\theta, \delta) = E_{X\sim P_\theta} E_{\delta \sim \lambda} l(\theta, \delta(X)) $$

  2. In section 3.3, they studied the relation between minimax rule and Bayes rule in terms of some two-player 0-1 game, where they introduced the nature and a statistician as the two players. The nature chooses a prior distribution $\pi$ over the parameter space $\Theta$, and the statistician chooses a randomized rule. The payoff one player pays to the other is

    $$ r(\pi, \delta) := E_{\theta \sim \pi} R(\theta, \delta) = E_{\theta \sim \pi} E_{\delta \sim \lambda} E_{X\sim P_\theta} l(\theta, \delta(X)) $$

    Q: Are $\theta$, the randomized rule $\delta$ and the sample supposed to be independent? I would like to know if we can exchange the order of the three expectations as:

    $$ r(\pi, \delta) = E_{\delta \sim \lambda} E_{\theta \sim \pi} E_{X\sim P_\theta} l(\theta, \delta(X)) $$

  3. The reason of my question in 2 is:

    They said that given $\delta$, $\arg\max_\pi r(\pi, \delta)$, when exists, can be chosen to the point mass probability measure on $\arg\max_\theta R(\theta, \delta)$. It means that, given a randomized rule $\delta$, the least favorite prior distribution $\pi_\delta$ on $\Theta$, can chosen to be a single $\theta_\delta$, i.e. we can remove the randomization of $\theta$

    Q: I would like to know if it is also similarly true that given $\pi$, the Bayes randomized rule $\arg\min_\delta r(\pi, \delta)$, when exists, can be chosen to have the point mass probability measure on $\arg\min_d E_{\theta \sim \pi} E_{X\sim P_\theta} l(\theta, d(X)) $?

    If yes, it implies a Bayes randomized rule is always a Bayes nonrandomized rule. But I am not sure if this conclusion is correct.

    I think that this can be true, if we can exchange the order of $ E_{\delta \sim \lambda}$ and $ E_{\theta \sim \pi}$ in $r(\pi, \delta) $ as in part 2, which is why I ask if $\theta$ and $\delta$ are independent in part 2.

Thanks.

The relevant pages are here for section 1.3 and here for section 3.3.

$\endgroup$
2
  • $\begingroup$ Thanks. @Glen_b. I have been trying to update my post for the past half an hour, but couldn't perhaps due to the site problem. $\endgroup$ – Tim Dec 1 '14 at 2:20
  • $\begingroup$ Indeed, I started experiencing problems a few minutes after I typed that comment. Whenever you can. $\endgroup$ – Glen_b Dec 1 '14 at 2:38
2
$\begingroup$
  1. Are the randomized rule δ and the sample supposed to be independent?

There is no reason to restrict randomised rules in this manner. For instance, randomisation may only occur for some values of the observation/sample.

  1. Are θ, the randomized rule δ and the sample supposed to be independent?

In a joint Bayesian model, sample and parameter are dependent since the sampling distribution is the conditional distribution. While the randomised rule cannot depend on the unknown $\theta$ in a functional manner, since it can [functionally] depend on the sample, it possibly does [probabilistically] depend on the unknown $\theta$.

  1. a Bayes randomized rule is always a Bayes nonrandomized rule

This is incorrect: A randomised Bayes rule may differ from a non-randomised one and achieve better performances from a risk perspective. For instance, in finite parameter spaces with strictly convex losses, a minimax rule is always a randomised Bayes rule but not necessarily a non-randomised Bayes rule

$\endgroup$
2
  • $\begingroup$ On your answer to #3, could you point me in the direction to a proof of this? Thanks. $\endgroup$ – John Nov 6 '19 at 13:35
  • $\begingroup$ This situation is covered in my book. $\endgroup$ – Xi'an Nov 7 '19 at 2:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.