Skip to main content
1 of 2
Mark L. Stone
  • 13.5k
  • 1
  • 38
  • 58

$\|x\|_2^2$ and $\|x\|_1$ are $L_p^p$ for p = 2 and 1 respectively, so they are consistent in that sense.

Nevertheless, there's no really standard convention on whether or not an L2 regularization term should be squared. In fact, there's not even standardization on whether the "baseline" portion of a "least squares" objective should be squared. You could have { $norm(Ax-b)$ or $norm(Ax-b)^2$ } plus a regularization term. In the absence of the regularization term, it doesn't matter, but with a regularization term, it does. So squaring or not the baseline portion of the objective and the regularization term (even if based on an $L_2$ norm) provides 4 different possible ways in effect of doing regularization, and although similar, they are all different.

And of course, we can change from the 2 norm to the 1 norm or something else in the baseline objective term $norm(Ax-b)$.

Bottom line: regularization tends to be somewhat of an ad hoc or arbitrary process, and there's more than one way to do it.

Mark L. Stone
  • 13.5k
  • 1
  • 38
  • 58