Squared errors under linear models are often preferred because of:
- the relation to orthogonality, that behaves well with respect to some random phenomena considered as noise (uncorrelatedness)
- it is convex and differentiable, not $L_1$
- it yields tractable optimization algorithms as the derivative turns into linear systems
$L_1$ is often considered as a convenient proxy or convex relaxation to the strict sparsity (the count of non-zero terms) which is combinatorially complicated, see for instance For Most Large Underdetermined Systems of Linear Equations the Minimal $\ell_1$-norm Solution is also the Sparsest Solution. Some tend to use $\ell_p$, $0<p<1$ to enforce more sparsity, at the cost of "losing" convexity.
However, the $\ell_0$ count measure is insensitive to non-zero scaling. Multiply a vector by a non-zero constant, the number of non-zeros terms will remain the same. Thus, $\ell_0$ is $0$-order homogeneous, while $\ell_p$ norms or quasi-norms are all $1$-order homogeneous. Even if, somehow, $\ell_p \to \ell_0$ as $p\to 0$, this discrepancy seems a gap to me.
Thus, keeping with norms, some are considering (non-convex) norm ratios, such as $\ell_1 / \ell_2 $, see for instance the references in Euclid in a Taxicab: Sparse Blind Deconvolution with Smoothed $\ell_1 / \ell_2 $ Regularization.