Could anyone provide some examples that a decision rule which is minimax but not Bayes, and a decision rule which is Bayes but not minimax? Thanks!

  • $\begingroup$ The two are based on different assumptions. Defining a minimax rule for a Bayesian setting would be odd. Similarly, a Bayesian rule with standard minimax assumptions would not be based on the assumptions and thus a touch odd. $\endgroup$ – Oxonon Jan 6 at 11:51
  • 1
    $\begingroup$ @Oxonon It doesn't matter whether this seems "odd," though, because the point is to understand the distinction between these approaches to making decisions and to appreciate the scope and limitations of theorems that indicate how to use Bayesian procedures to find minimax solutions. Bayes procedures are a basic tool even in classical ("frequentist") statistical theory. There is no contradiction of assumptions involved. $\endgroup$ – whuber Jan 6 at 14:11
  • $\begingroup$ @whuber, Surely you either start with a hypothesis space and a statement to the effect of "the true parameter lies in this space" or you start with "My belief is that the true parameter has some given distribution". And from there you act optimally with respect to the given information. Stating that you can find the minimax solution by setting up and solving a different appropriate Bayes problem is still just a method of finding the minimax solution --- there is no natural Bayes solution for that problem. Thatt is to say, the solution is determined by the assumptions, not method. $\endgroup$ – Oxonon Jan 6 at 15:55
  • 1
    $\begingroup$ @Oxonon Right. This question is about decision rules, not interpretation or statistical practice. $\endgroup$ – whuber Jan 6 at 16:54

In the event the decision problem "has a value", i.e., when minimax meets maximin, $$\min_\delta\,\max_\theta\,\mathbb E_\theta[L(\theta,\delta(X)]= \max_\pi\min_\delta\mathbb \int \mathbb E_\theta[L(\theta,\delta(X)] \pi(\text{d}\theta)$$ there exists a Bayes procedure or a limit of Bayes procedures that is minimax. (This is actually a constructive method for producing minimax estimators in difficult settings: seek a Bayesian estimator with minimax risk, as eg in 1981 Casella's and Strawderman's derivation of the [unique] minimax estimator of a bounded Normal mean.) However, this intersection between both concepts does not mean they are confounded.

For examples of minimax procedures that are not Bayes or limits of Bayes procedures, a classical example is the class of James-Stein estimators: when estimating a Normal mean vector $\theta\in\mathbb R^p$ with $p\ge 3$ and a quadratic loss function$$L(\theta,\delta)=(\delta-\theta)^\text{T}A(\delta-\theta)$$with $A$ a $p\times p$ symmetric positive definite matrix, estimators of the form$$\delta\,: x \longmapsto \delta(x)=\left\{ 1 - {a}||x||^{-2}\right\}x$$are minimax if $0\le a\le \bar{a}(p,A)$ where the upper bound is a function of the dimension $p$ and of the matrix $A$. For instance, $\bar{a}(p,\mathbf I_p)=2(p-2)$.

Conversely, most Bayes procedures are not minimax. Indeed, any prior $\pi$ such that$$\min_\delta\,\max_\theta\,\mathbb E_\theta[L(\theta,\delta(X)]= > \min_\delta\mathbb \int \mathbb E_\theta[L(\theta,\delta(X)] \pi(\text{d}\theta)$$will not produce a minimax Bayes procedure.

Here is an illustration from my book:

Consider a Bernoulli observation, $x\sim \mathcal B e(\theta)$ with $\theta\in\{0.1,0.5\}$. Four nonrandomized \est s of $\theta$ are available, \begin{eqnarray*} \delta_1(x) & = & 0.1,\qquad \qquad \delta_2(x) = 0.5, \\ \delta_3(x) & = & 0.1\, \mathbb I_{x = 0} + 0.5\, \mathbb I_{x = 1}, \quad \delta_4(x) = 0.5\, \mathbb I_{x = 0} + 0.1\, \mathbb I_{x = 1}. \end{eqnarray*} Assume that the penalty for a wrong answer is $2$ when $\theta = 0.1$ and $1$ when $\theta = 0.5$. The risk vectors $(R(0.1,\delta),R(0.5,\delta))$ of the four estimators are then, respectively, $(0,1)$, $(2,0)$, $(0.2,0.5)$, and $(1.8,0.5)$. It is straightforward to see that the risk vector of any randomized estimator is a convex combination of these four vectors or, equivalently, that the risk set, $\mathfrak R$, is the convex hull of the above four vectors, as represented by the following Figure

enter image description here

In this case, the minimax estimator is obtained at the intersection of the diagonal of $\mathbb R^2$ with the lower boundary of $\mathfrak R$. As shown by this Figure, this estimator $\delta^*$ is randomized and takes the value $\delta_3(x)$ with probability $\alpha = 0.87$ and $\delta_2(x)$ with probability $1-\alpha$. The weight $\alpha$ is actually derived from the equation $$0.2 \alpha + 2(1-\alpha) = 0.5 \alpha. $$ This estimator $\delta^*$ is also a (randomized) Bayes estimator with respect to the prior $$\pi(\theta) = 0.22\, \mathbb I_{0.1}(\theta) + 0.78\, \mathbb I_{0.5} (\theta); $$ the prior probabilities $\pi_1 = 0.22$ corresponds to the slope between $(0.2,0.5)$ and $(2,0)$, i.e., $$ {\pi_1\over 1-\pi_1} = {0.5 \over 2-0.2}. $$ Notice that every randomized estimator that is a combination of $\delta_2$ and of $\delta_3$ is a Bayes estimator for this distribution, but that $\delta^*$ only is also a minimax estimator.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.