Admissible Estimator for Linear Regression

Question

Is there an admissible estimator for a linear regression model with many parameters without restricting the parameter space?

Admissibility will be with respect to Mean Square Error on the regression parameter vector.

It seems that James-Stein estimators beat OLS, but are still inadmissible.

Thanks

Answer 1

[I will use the $x\sim\mathcal{N}_p(\mu,I_p)$ case instead of the equivalent linear regression model in order to simplify notations.]

Since the family of James-Stein estimators$$\delta_a(x)=\left(1-\frac{a}{||x||}\right)^+x$$all have a strictly smaller mean square error (risk) than $\delta_0$ when $0<a<2(p-2)$, $\delta_0$ is not admissible. While the $\delta_a$ themselves are also inadmissible because they are not analytic functions of $x$, they are dominated by admissible estimators that are either proper Bayes or limits of proper Bayes estimators, by virtue of the complete class theorem (see, e.g., Section 8.3.3 of my book). Hence $\delta_0$ is inadmissible and dominated by admissible estimators. For instance, see Maruyama (2004) for a class of admissible minimax (hence dominating $\delta_0$) estimators. And Berger and Robert (1990), and Maruyama and Strawderman (2006) for classes of minimax generalised Bayes estimators.

Here is the main complete class result:

Theorem 8.3.9 (Stein's necessary and sufficient condition) Under the hypotheses

1. $f(x|\theta)$ is continuous in $\theta$ and strictly positive on $\Theta$; and
2. the loss $\rm{L}$ is strictly convex, continuous and, if $E\subset\Theta$ is compact,$$\lim_{\|\delta\|\rightarrow +\infty} > \inf_{\theta\in E} \rm{L}(\theta,\delta) =+\infty$$

an estimator $\delta$ is admissible if, and only if, there exist a sequence $(F_n)$ of increasing compact sets such that $\Theta=\bigcup_n F_n$, a sequence $(\pi_n)$ of finite measures with support $F_n$, and a sequence $(\delta_n)$ of Bayes estimators associated with $\pi_n$ such that

1. there exists a compact set $E_0\subset \Theta$ such that $\inf_n \pi_n(E_0) \ge 1$
2. if $E\subset \Theta$ is compact, $\sup_n \pi_n(E) > <+\infty$
3. $\lim_n r(\pi_n,\delta)-r(\pi_n) = 0$
4. $\lim_n R(\theta,\delta_n)= R(\theta,\delta)$.

Answer 2

Well if you only care about your estimator being admissible, any trivial estimator will do the job...

If you also want convergence, true admissiblity seems pretty complicated. I think asymptotical admissibility is the best you can get (look for results like Pinsker's theorem).