Mean Absolute Error (MAE) derivative $$MAE=|y_\textrm{pred} - y_\textrm{true}|$$
$$\dfrac{\mathrm dMAE}{\mathrm dy_\textrm{pred}} = ?$$
I'm trying to understand how MAE works as a loss function in neural networks using backpropogation. I know it can be used directly in some APIs - e.g. Keras - however I see tensorflow doesn't allow it (although you can manually declare it tf.abs(tf.minus(y_pred,y_true))).
My question is: How is the derivative of MAE generally calculated (I only found this which uses an apparently complex approximation) and specifically how is it computed in tensorflow (when manually declaring) and keras?
 A: The mae, as a function of $y_{\text{pred}}$, is not differentiable at $y_{\text{pred}}=y_{\text{true}}$. Elsewhere, the derivative is $\pm 1$ by a straightforward application of the chain rule:
$$\dfrac{d\text{MAE}}{dy_{\text{pred}}} = 
\begin{cases}
  +1,\quad y_{\text{pred}}>y_{\text{true}}\\
  -1,\quad y_{\text{pred}}<y_{\text{true}}
\end{cases}$$
The interpretation is straightforward: if you are predicting too high ($y_{\text{pred}}>y_{\text{true}}$), then increasing $y_{\text{pred}}$ yet more by one unit will increase the MAE by an equal amount of one unit, so the gradient encourages you to reduce $y_{\text{pred}}$. And vice versa if $y_{\text{pred}}<y_{\text{true}}$.
Skimming the paper you link, it seems like they approximate the MAE by a differentiable function to avoid the "kink" at $y_{\text{pred}}=y_{\text{true}}$.
As to what specifically is implemented in TensorFlow and keras, that is off-topic here. Best to consult the documentation, the source code or any specific help community.
A: As for Tensorflow's Keras here is the good answer:
TensorFlow or Theano: how do they know the loss function derivative based on the neural network graph?
And if we will dig into the source code we will see that it uses math_ops.sign wich is return 0 for 0:
https://github.com/tensorflow/tensorflow/blob/c91e944d626b517781af6a63c0aee302ab2457e3/tensorflow/python/ops/math_grad.py#L577
A: In this way, we can use Subgradient method
Subgradient descent
$$w^{t+1}=\ w^t-\alpha g$$
$$g=\begin{cases}1& x> 0 \\ [-1,1] & x=0\\-1 & x<0\end{cases}$$
Derivation of MAE with L1 and L2 regularization
$$MAE=\frac{1}{n}\sum_{i=0}^{n}\left|y_{pred}-y_{true}\right| \,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,{(\left|u\right|)}^′=\frac{uu^′}{\left|u\right|}$$






L1
L2




$$MAE=\frac{1}{n}\sum_{i=0}^{n}\left|\left(w_1x+w_0\right)-y^i\right|$$
+
$$\frac{\lambda}{n}\sum_{i=1}^{n}\left|w_j\right|$$
$$\frac{\lambda}{n}\sum_{i=1}^{n}w_j^2$$


$$\frac{dMAE}{dw_1}=\frac{1}{n}\sum_{i=0}^{n}{\frac{w_1x+w_0-y^i}{\left|\left(w_1x+w_0\right)-y^i\right|}\left(x\right)}$$
+
$$\lambda\frac{w_j}{\left|w_j\right|}$$
$$2\lambda w_j$$


$$\frac{dMAE}{dw_0}=\frac{1}{n}\sum_{i=0}^{n}{\frac{w_1x+w_0-y^i}{\left|\left(w_1x+w_0\right)-y^i\right|}\left(1\right)}$$
+
-
-



