1
$\begingroup$

I'm trying to implement a simple neural network (in Python) with 1 hidden layer and a loss function based on the GINI coefficient. But I just cannot find a way to minimize it with gradient descent. I'm not sure if this is even possible with this method. A simplified form can be written this way with e, current vector of errors : $$ e = y - \hat y $$ $$min\ G(e)=min\sum\limits_i\sum\limits_j \mid e_i-e_j\mid$$

Any idea or a trick to do this ? Thanks

$\endgroup$
5
  • 3
    $\begingroup$ What exactly are you stuck on? The loss looks (almost everywhere) differentiable to me, so what is the problem? $\endgroup$
    – shimao
    Mar 6 '18 at 21:26
  • $\begingroup$ Im trying to get the derivative but im stuck due to the presence of absolute values. $\endgroup$
    – tkon tkon
    Mar 6 '18 at 21:51
  • $\begingroup$ The derivative of $f(x)=|x|$ is $1$ if $x>0$ and $-1$ if $x<0$ and undefined at $x=0$ (look at a plot of $f$). Use the chain rule. $\endgroup$ Mar 8 '18 at 7:30
  • $\begingroup$ @StephanKolassa I've copied your comment as an answer. If you'd like to write your own answer, please let me know so that I can delete mine. $\endgroup$
    – Sycorax
    Jul 5 '18 at 15:02
  • 1
    $\begingroup$ @Sycorax: thanks! I'll go ahead and upvote your answer. And I'll stop answering in comments. $\endgroup$ Jul 5 '18 at 19:22
2
$\begingroup$

In comments, @StephanKolassa writes

The derivative of $f(x)=|x|$ is $1$ if $x>0$ and $−1$ if $x<0$ and undefined at $x=0$ (look at a plot of $f$). Use the chain rule.


I've copied this comment as a community wiki answer because the comment is, more or less, an answer to this question. We have a dramatic gap between answers and questions. At least part of the problem is that some questions are answered in comments: if comments which answered the question were answers instead, we would have fewer unanswered questions.

Please review

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.