# How is lasso an M-Estimator?

The definition of an M-estimator is an estimator (from Casella and Berger) of the form $$\hat{\theta}=\min \sum_{i=1}^n \rho(X_i-\theta),$$

where $X_1,X_2, \cdots, X_n$ is the data for some function $\rho$.

The LASSO estimate is defined as $$\hat{\theta}=\min ||AX-\theta||_2^2 + \lambda ||\theta||_1,$$ for some matrix A, and a vector parameter $\theta$.

Many papers refer to LASSO as an M-estimator, which does not appear to be true. Am I missing something here? The definition seems to imply there can not be any M-estimator for a vector parameter.

I am not familiar with LASSO being referred to as a conventional M-Estimator, as the conventional M-Estimator does not have a constraint, leading to familiar unconstrained optimization problems. I think LASSO is generally referred to as a regularized M-Estimator, i.e. an M-estimator with a regularity parameter. This would be $\lambda$ on the $\mathcal{L}_1$ norm or (LASSO) $\mathcal{L}_2$ norm (Ridge). See here for a nice description of the problem.
• Thanks, but even $\min ||AX-\theta||_2^2$$does not appear to be an M-estimator by the definition, right? – Devil Sep 11 '14 at 20:28 • The definition you supply in the question doesn't appear to be correct. That would make quasilikelihood and GEE not a form of M estimation which isn't the case. – AdamO Sep 11 '14 at 22:12 • You are right. Many other papers like www4.stat.ncsu.edu/~boos/papers/mest6.pdf seem to define an M-estimator differently. They define an M-estimator as an estimator obtained by solving equations of the form$\sum_{i=1}^n \psi(X_i,\theta)=0\$ as an M-estimator, in which case even least squares is an M-estimator. Thank you. – Devil Sep 12 '14 at 18:27