31
$\begingroup$

I am new to optimization. I keep seeing equations that have a superscript 2 and a subscript 2 on the right-hand side of a norm. For instance, here is the least squares equation

min $ ||Ax-b||^2_2$

I think I understand the superscript 2: it means to square the value of the norm. But what is the subscript 2? How should I read these equations?

$\endgroup$
4
  • 4
    $\begingroup$ $||\theta||_p$ is the $\ell_p$-norm of $\theta$. Let's say $\theta$ is $d$-dimensional, then $||\theta||_p = \left(\sum_{i=1}^d |\theta_i|^p\right)^\frac{1}{p}$. $\endgroup$
    – Sobi
    Commented Dec 15, 2015 at 17:21
  • $\begingroup$ Single vertical bars are used for absolute value (magnitude): $|\theta|$ $\endgroup$ Commented Dec 15, 2015 at 17:28
  • $\begingroup$ Thanks!...but what is the superscript 2 for?...the subsript is for the pth norm....the superscript is for? $\endgroup$
    – mathopt
    Commented Dec 15, 2015 at 18:37
  • $\begingroup$ @user1467929: Squaring - if it's anything else they'd surely have said. $\endgroup$ Commented Dec 15, 2015 at 19:48

2 Answers 2

35
$\begingroup$

You are right about the superscript. The subscript $||.||_p$ specifies the $p$-norm.

Therefore:

$$||x_i||_p=(\sum_i|x_i|^p)^{1/p}$$

And:

$$||x_i||_p^p=\sum_i|x_i|^p$$

$\endgroup$
3
  • $\begingroup$ ah. And there are conventions for the meanings of the subscripts I see. en.wikipedia.org/wiki/Norm_(mathematics)#p-norm. So like 1 = taxicab norm, 2=euclid norm etc $\endgroup$
    – bernie2436
    Commented Nov 13, 2015 at 14:04
  • $\begingroup$ @bernie2436: These are special cases of the general definition given in the answer above (except maybe the sup-norm with $p = \infty$) $\endgroup$
    – Michael M
    Commented Nov 13, 2015 at 14:55
  • $\begingroup$ This is a good explanation. $\endgroup$
    – StatsBio
    Commented Sep 20, 2022 at 11:29
22
$\begingroup$

$\|x\|_2$ is the Euclidean norm of the vector $x$; $\|x\|_2^2$ is the squared Euclidean norm of $x$. Note that as the Euclidean norm is probably the mostly commonly used norm people routinely abbreviated by $\|x\|$. By definition when assuming a Euclidean vector space: $\|x\|_2 := \sqrt{x_1^2 + x_2^2 + \dots + x_n^2}$.

As mentioned in the comments, the subscript $p$ refers to the degree of the norm. Other commonly used norms are for $p = 0$, $p = 1$ and $p = \infty$. For $p=0$ one gets the number of non-zero elements in $x$, for $p=1$ (ie. $\|x\|_1$) one gets the Manhattan norm and for $p = \infty$ one gets the maximum absolute value from the elements in $x$. Both $p = 0$ and $p = 1$ are popular in sparse/compressed application settings where one wants to "urge" some coefficient(s) to be zero.

$\endgroup$
0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.