Let $r(\pi, \delta)$ denote the Bayes risk of an estimator $\delta$ with respect to a prior $\pi$, let $\Pi$ denote the set of all priors on the parameter space $\Theta$, and let $\Delta$ denote the set of all (possibly randomized) decision rules.

The statistical interpretation of John von Neumann's minimax inequality states that

$$ \sup_{\pi\in\Pi} \inf_{\delta\in\Delta} r(\pi, \delta) \leq \inf_{\delta\in\Delta}\sup_{\pi\in\Pi} r(\pi, \delta), $$

with strict equality guaranteed for some $\delta'$ and $\pi'$ when $\Theta$ and $\Delta$ are both finite.

Can someone provide a concrete example where the inequality is strict?


An example of strict von Neumann inequality occurs when the risk function $r$ satisfies the following conditions for some values $r_0 < r_1$ (where the former value is "low" and the latter is "high"):

$$\begin{matrix} \forall \pi \in \Pi, \exists \delta \in \Delta: & & r(\pi, \delta) = r_0, & & (1) \\[8pt] \forall \delta \in \Delta, \exists \pi \in \Pi: & & r(\pi, \delta) = r_1. & & (2) \end{matrix}$$

The first condition says that regardless of the prior, there is always a decision rule with low risk $r_0$, which gives $\sup_{\pi\in\Pi} \inf_{\delta\in\Delta} r(\pi, \delta) = r_0$. The second condition says that regardless of the decision rule there is always some prior giving high risk $r_1$, which gives $\inf_{\pi\in\Pi} \sup_{\delta\in\Delta} r(\pi, \delta) = r_1$.

Another way of stating this situation is that there is no decision rule (chosen before seeing the prior) that guarantees low risk for every prior (sometimes it will have high risk), but for every prior, there is some decision rule (chosen after seeing the prior) that guarantees low risk. In other words, in order to impose a low bound on the risk we need to adapt our decision rule to the prior.

Example: A simple example of this kind of situation occurs when you have a pair of allowable priors $\pi_0, \pi_1$ and a pair of allowable decision rules $\delta_0, \delta_1$ with a risk matrix like this:

$$\begin{array}{ll} r(\pi_0, \delta_0) = r_0 & & r(\pi_1, \delta_0) = r_1, \\[6pt] r(\pi_0, \delta_1) = r_1 & & r(\pi_1, \delta_1) = r_0. \\[6pt] \end{array}$$

In this case there is no decision rule that guarantees low risk over both priors, but for each prior there is a decision rule that has low risk. This situation satisfies the above conditions which gives strict inequality in the von Neumann inequality.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.