From this page and in this paper (first paragraph of chapter 2.1) there is the term of "$l_1$-normalization" or absolute normalization of a vector (i.e. some data). The scope is to turn the data into a distribution (PMF) that sums up to 1. The definition on the page is:

It may be defined as the normalization technique that modifies the dataset values in a way that in each row the sum of the absolute values will always be up to 1. It is also called Least Absolute Deviations.

For example $v=[1,2,3]^T$. Does the $l_1$-normalization simply mean: $$ l_1(v_i)=\frac{|v_i|}{\sum_{j=1}^n |v_j|} \Leftrightarrow l_1(v)=[\frac{1}{6},\frac{2}{6},\frac{3}{6}]^T $$ ? Also, is this "better" in terms of keeping the original proportions than the min-max or softmax normalization?


In general, the normalization of a vector $v$ with respect to a norm $\| \cdot \|$ is given by $y = \frac{v}{\| v \|}$. This new vector $y$ has the properties that (1) it has norm one, meaning that $\| y \| = 1$, and (2) it has the same direction as the original vector $v$, meaning that $v$ is proportionate to $y$.

The term $\ell_1$ normalization just means that the norm being used is the $\ell_1$ norm $\| v \|_1 = \sum_{i = 1}^n |v_i|$. This means that your formula is somewhat mistaken, as you shouldn't be taking the absolute values of the $v_i$'s in the numerator. And note that in general, $\ell_1$ normalization does not make a vector into a pmf because the normalized vector can have negative entries.

Vector normalization always preserves the original proportions, regardless of the norm used, because $$ \frac{v_i}{v_j} = \frac{v_i / \| v \|}{v_j / \| v \|}, $$ but min-max and softmax normalizations do not. Try it with the vector $v = (1, 2, 3)$:

$$\text{min-max}(v) = \left( \frac{1 - 1}{3 - 1}, \frac{2 - 1}{3 - 1}, \frac{3 - 1}{3 - 1}\right) = \left(0, \frac{1}{2}, 1\right) \\ \text{softmax}(v) = (0.09, 0.24, 0.67) $$ which clearly don't have the same proportions as $(1, 2, 3)$.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.