# What is a sparse estimator?

Can anyone point me to a reference, either book or paper, where I can find the precise definition of sparse estimator?

Thanks

-
Not sure, but maybe sparsity of an estimator (vector valued) $\hat{\theta}$ means that most off-diagonal entries of the covariance matrix are zeros, so components of $\hat{\theta}$ are mostly orthogonal. Maybe you should look for "orthogonal parametrization" in the literature. –  Matteo Fasiolo Nov 22 '12 at 13:49

Sparse estimators are frequently used in a high dimensional context, namely $p>>n$. Essentially, they offer a regularized version of an estimator e.g. a least squares estimator with an $l_1$ or $l_0$ norm based parameter penalty.
Assume you estimate the projected version of $y$ in a subspace of $p$ dimension spanned by the columns of $\mathbf{X}$ given by $\hat{y}= \mathbf{X}\beta$. This equation defines a hyperplane as $\{\beta \in R^p |\mathbf{X}\beta=\hat{y}\}$. Imposing an $l1$ penalty on the allowed values of $\beta$ we define 1-norm ball see figure .
The distance in $l1$ is achieved when the boundary of the ball meets the hyperplane, the solution set will have a minimal cardinality compared to the spherical nature of the $l2$ norm.
In a two dimensional plane this can occur to y-axis which implies a solution of the form $\hat{\beta}=\{0,\hat{\beta_y}\}$, a kind of variable selection.