# Minimization of expected risk

On page 9 of "High Dimensional Sparse Econometric Models: An Introduction (2011)," Belloni and Chernozhukov explain in Remark 1 that the expected risk of a sparse estimator is $$\min_{\beta\in\mathbb{R}^{\tilde{T}}} \mathbb{E}_{n}[(f_{i}-x_{i}[\tilde{T}]'\beta)^{2}]+\sigma^{2}\dfrac{k}{n}.$$ The notation is explained in the paper, and is standard. $x_{i}[\tilde{T}]$ denotes a vector of values (say, $k$) of some subset of predictors, generally smaller than $p$, the number of all potential predictors. I fail to understand why. Help would be appreciated. Here is direct arXiv link to the paper.

• Since $\beta$ is a vector of length p ($\beta \in \mathbb{R}^p$) then necessarily $x_i[\tilde T]$ is a vector of length p as well, or their inner product could not be formed. Could you clarify your question? – jwimberley Jan 14 '17 at 21:41
• You are right, that was a typo on my part. Corrected. – rsm Jan 14 '17 at 21:51

Let $\Pi$ a projection matrix. Then, \begin{align*} \frac{1}{n} \, \mathbb{E} \| \Pi y - f^* \|_2^2 & = \frac{1}{n} \, \mathbb{E} \|\Pi f^* - f^* - \Pi \epsilon \|_2^2 \\ & = \frac{1}{n} \, \mathbb{E} \|\Pi f^* - f^*\|_2^2 + \frac{1}{n} \, \mathbb{E} \|\Pi \epsilon \|_2^2 \\ & = \frac{1}{n} \, \mathbb{E} \|\Pi f^* - f^*\|_2^2 + \frac{1}{n} \, \mathbb{E} \, \textrm{trace} \left( \|\Pi \epsilon \|_2^2 \right) \\ & = \frac{1}{n} \, \mathbb{E} \|\Pi f^* - f^*\|_2^2 + \frac{1}{n} \, \mathbb{E} \, \textrm{trace} \left( \epsilon^T \Pi \epsilon \right) \\ & = \frac{1}{n} \, \mathbb{E} \|\Pi f^* - f^*\|_2^2 + \frac{1}{n} \, \mathbb{E} \, \textrm{trace} \left( \epsilon \epsilon^T \Pi \right) \\ & = \frac{1}{n} \, \mathbb{E} \|\Pi f^* - f^*\|_2^2 + \frac{\sigma^2}{n} \, \textrm{trace} \left( \Pi \right) \\ & = \frac{1}{n} \, \mathbb{E} \|\Pi f^* - f^*\|_2^2 + \frac{\sigma^2}{n} k, \\ \end{align*} where $k$ is the dimension of the space onto which $\Pi$ is projecting.
This is the "bias$^2$ + variance" decomposition of the risk, and it appears to be the computation behind their equation.
I should note that this computation only reveals that this decomposition holds when the estimate $x^T \beta$ is a projected form of the signal $f^*$---however, this won't necessarily be the case, since they're using the form $\Pi f^* = x^T \beta$ for an arbitrary $\beta$ (which they then minimize over.)