# Why is the maximum risk of an estimator independent of a prior distribution over the parameter?

One way of choosing an estimator $$\delta(x)$$ for data $$X$$ distributed as $$P_{\theta}(X)$$, where $$\theta \in \Theta$$ is:

$$minimize \sup_{\theta \in \Theta} Risk(\delta(x), \theta)$$

In this case why is this risk independent of a prior over $$\theta$$?

• @Xi'an I edited the post to a prior rather than the prior – wabbit Apr 14 at 18:29

The approach in the question is called minimaxity, which compares estimators in terms of their worst performance, $$\sup_{\theta\in\Theta} R(\delta,\theta)=\sup_{\theta\in\Theta} \Bbb E_\theta[L(\delta(X),\theta)]$$ Since this quantity is a real number all estimators become comparable under this criterion and hence there may exist a solution to the minimisation program $$\inf_\delta \sup_{\theta\in\Theta} R(\delta,\theta)$$ At no stage in this formalisation does a prior occur because (a) estimators can be constructed from any point of view, not necessarily Bayesian, and (b) the distribution involved in the computations is purely frequentist, i.e. a distribution in $$X$$. Further, considering the supremum in $$\theta$$ cancels the relevance of a prior distribution over $$\theta$$.
Note however that in regular problems, maximin and minimax coincide (see e.g. in my book, Section 2.4.3), that is $$\inf_\delta \sup_{\theta\in\Theta} R(\delta,\theta)=\sup_\pi\inf_\delta \int R(\delta,\theta)\,\pi(\theta)\text{d}\theta.$$This means that, when the problem is regular enough, the minimax estimator is also Bayes against a prior called least favourable prior.
Consider $$x\sim{\mathcal B}(n,\theta)$$ when $$\theta$$ is to be estimated under the quadratic loss, $$\mathrm L (\theta,\delta) = (\delta-\theta)^2.$$ Bayes estimators are then given by \post\ expectations (see Section 2.4) and, when $$\theta \sim{\mathcal B}e \left(\frac{\sqrt{ n}}{2}, \frac{\sqrt{ n}}{2} \right)$$ the posterior mean is $$\delta^* (x) = \frac{x+ \sqrt{n}/2}{n+ \sqrt{ n}}.$$ Moreover, this estimator has constant risk, $$R(\theta,\delta^*) = 1/4(1+\sqrt{n})^2$$ Therefore, integrating out $$\theta$$, $$r(\pi) = R(\theta,\delta^*)$$ and this shows $$\delta^*$$ is minimax according to Lemma 2.4.13. Notice the difference with the maximum likelihood estimator, $$\delta_0(x) = x/n$$ for small values of $$n$$, and the unrealistic concentration of the prior around $$0.5$$ for larger values of $$n$$.