Should the existence of a closed-form solution inform the choice of robust regression method? Suppose one has a linear least squares problem of the form \begin{align} \xi^* = \textrm{arg min} \ \sum_{i = 0}^n \left \lvert  \ {\bf v}^T({\bf x}_i) \ \xi - c({\bf x}_i)  \ \right \rvert^{\ 2}  \in \mathbb{R}^m, \ 6 \leq m \leq 12  \end{align} 
where ${\bf x}_i \in \mathbb{R}^3$, ${\bf v}({\bf x}_i) \in \mathbb{R}^m$,   and $c({\bf x}_i) \in \mathbb{R}$. 
Under the $\ell^2$ norm as written, the problem has a closed form solution given by $$ \xi^* = {\cal M}^{-1} {\boldsymbol{ \cal b}}, $$ where \begin{gather*} {\cal M} = \sum_{i}^n {\bf v} {\bf v}^T ({\bf x}_i) \in \mathbb{R}^{m \times m} \\
\boldsymbol{\cal{b}} = \sum_{i=0}^n c({\bf x}_i) \ {\bf v}({\bf x}_i) \in \mathbb{R}^m.
\end{gather*}
Furthermore, suppose that the number of points, $n$, is such that $$ 2^{15} \leq n$$ and that the data is generally noisy, to the degree you would expect a 3D point cloud corresponding to the projection of a depth map from a stereo camera to be.
This number of outliers are significant enough to the point where the linear least squares problem does a relatively poor job of recovering $\xi^*$ and a robust regression method is required for an accurate solution.
My question boils down to this:
Are some robust regression methods particularly well-suited for problems that when initially formulated under the $\ell^2$ norm, have closed form solutions?
 A: So you want to minimize absolute error instead of squared error, I take it. I don't think you're going to find something with an analytical solution, but Support Vector Regression does what you want, I think, i.e. outliers do not exert "undue" influence on the slope of the regression line.
A: Supposing the 'original' least squares problem has a closed form solution, I'm not aware of a reason why this would affect the quality of any robust regression methods relative to others. However, it would reduce the computational cost of robust methods that involve repeatedly solving least squares problems at each step. For example, this would be true for M-estimation (using the iteratively reweighted least squares algorithm) and RANSAC.
In any case, you have a standard linear regression problem, which is the most commonly investigated setting for robust regression. So, many robust methods are available to choose from. Keep in mind that robust regression problems themselves typically don't have closed form solutions, even for linear regression.
