Usually, people use L2 norm as a machine learning error function.

per wiki

the error function for a Perceptron model is

${\displaystyle {\frac {1}{s}}\sum _{j=1}^{s}|d_{j}-y_{j}(t)|}$

why is that?


Note that the perceptron is a binary classifier, so its output $d_j$ for each input will be in $\{0,1\}$, and it will be compared to the known value $y_j$ which is also in $\{0,1\}$. Convince yourself that in this scenario L1 loss and L2 loss are the same. And since the original, classical single layer perceptron algorithm doesn't involve taking derivatives, it doesn't matter that the loss function has differentiability issues.

Successors of the perceptron used gradient descent to update the weights in search of the "best" separating hyperplane, and they indeed define loss with respect to L2. Here's an example: https://sebastianraschka.com/Articles/2015_singlelayer_neurons.html#gradient-descent


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.