In James-Stein's estimator we have a $p$-dimensional random vector $X\sim N_{p}(\mu ,I)$ where $\mu \neq 0$ and the goal is to estimate the mean vector using the single ($n=1$) data vector $X$. The squared loss of any estimator $\hat{\mu}$ is measured: \begin{equation*} L(\hat{\mu},\mu)=\sum_{i=1}^{p}(\mu_{i} - \hat{\mu}_{i})^{2}=\| \mu-\hat{\mu} \|_{2}^{2} \end{equation*} This leads to risk: \begin{equation*} R(\hat{\mu})=\mathop{\mathbb{E}} \| \hat{\mu}-\mu \| ^{2} \end{equation*} The risk of the Maximum Likelihood Estimator (MLE) $\hat{\mu}_{MLE}=X$: \begin{equation*} R(\hat{\mu}_{MLE})=\sum_{i=1}^{p}\mathop{\mathbb{E}}(X_{i}-\mu_{i})^{2}=p\cdot var(X_{1})=p \end{equation*} I read this book: High‐Dimensional Covariance Estimation. In page 30 of this book, example 7, the author shrinks the MLE toward the zero. For doing this, the author finds $\beta_{0}$ the minimizer of $R(\mu_{\beta})$ where $\mu_{\beta}=\beta X$ : \begin{equation} \beta_{0}=(1+p/\| \mu \|^{2})^{-1} \end{equation} I do not understand the calculations done to get the $\beta_{0}$. Can someone explain how to find $\beta_{0}$?
1 Answer
The risk of $\mu_\beta$ is \begin{align} R(\mu_\beta) &= \sum_{i=1}^{p}\mathop{\mathbb{E}}(\beta X_{i} - \mu_{i})^2 \\ &= \sum_{i=1}^{p} \mathop{\mathbb{E}}( \beta^2 X_{i}^2 - 2\beta X_i \mu_i + \mu_{i}^2) \\ &= \sum_{i=1}^{p} \beta^2 (1 + \mu_i^2) - 2\beta \mu_i^2 + \mu_{i}^2 \\ &= a\beta^2 + b\beta + c, \end{align} where $a = \sum_{i=1}^{p} (1 + \mu_i^2) > 0$, $b = - 2\sum_{i=1}^{p} \mu_i^2$, and $c = \sum_{i=1}^{p} \mu_i^2$. The risk is minimized when $\frac{d}{d\beta} R(\mu_\beta) = 2a\beta + b = 0$, i.e. for \begin{align} \beta_0 &= \frac{-b}{2a} \\ &= \frac{\sum_{i=1}^{p} \mu_i^2}{\sum_{i=1}^{p} (1 + \mu_i^2)} \\ &= \frac{\| \mu \|^2}{p + \| \mu \|^2} \\ &= \left(\frac{p + \| \mu \|^2}{\| \mu \|^2}\right)^{-1} \\ &= \left(1 + \frac{p }{\| \mu \|^2}\right)^{-1}. \\ \end{align}
