# Kernel and regularization parameter of James–Stein estimator

Consider a FIR model of the form $y= Ug_0+e$ with $e$ white noise with variance $\sigma^2$. We assume that we have collected N input-output measurements $y$ and $U$. The James–Stein estimator is defined as $$\hat{g}_{JS}= \Big(1-\frac{(n-2)\sigma^2}{||U\hat{g}_{LS}||^2}\Big)\hat{g}_{LS}$$

where $\hat{g}_{LS}$ is the Least Square estimator. Knowing that $\hat{g}_{LS}= (U^TU)^{-1}U^Ty$, is it possible to write the JS estimator as in the form below? $$\hat{g}_{JS}= (U^TU + \gamma_{JS}(K_{JS})^{-1})^{- 1}U^Ty$$ I am trying to compare the two estimators and I would like to have them written in comparable formulation. In principle, few analytical steps are necessary to rearrange the first formulation into the second. So, the kernel $K_{JS}$ and the regularization parameter $\gamma_{JS}$ are derived as \begin{align} & \Big( 1 -\frac{(n-2)\sigma^2}{||U\hat{g}_{LS}||^2} \Big)\hat{g}_{LS}= (U^TU+\gamma_{JS}(K_{JS})^{-1})^{-1}U^Ty \\ & \Big( 1 -\frac{(n-2)\sigma^2}{||U(U^TU)^{-1}U^Ty||^2} \Big)(U^TU)^{-1}= (U^TU+\gamma_{JS}(K_{JS})^{-1})^{-1} \end{align} Note that $||U(U^TU)^{-1}U^Ty||^2= ||y||^2$, where $||y||^2= ||Ug_0+e||^2= g_0^TU^TUg_0 + \sigma^2$, so we can write \begin{align} & U^TU+\gamma_{JS}(K_{JS})^{-1} = \Big( 1 -\frac{(n-2)\sigma^2}{ g_0^TU^TUg_0 + \sigma^2} \Big)^{-1} (U^TU)\\ & \gamma_{JS}(K_{JS})^{-1} = -U^TU + \Big( 1 -\frac{(n-2)\sigma^2}{ g_0^TU^TUg_0 + \sigma^2} \Big)^{-1} (U^TU)\\ & \gamma_{JS}(K_{JS})^{-1} = U^TU \Big[ \Big( 1 -\frac{(n-2)\sigma^2}{ g_0^TU^TUg_0 + \sigma^2} \Big)^{-1} -1\Big] \end{align} Are there any ways to make this formulation look "nicer"?

• This is a very interesting question. Is it possible to write JS as a linear smoother? – Cagdas Ozgenc May 30 '19 at 18:49