6
$\begingroup$

I have seen people speaking of the Log-Cosh Loss that is 2 times differentiable and mimic the Mean Absolute Error goers 0. It is therefore useful for algorithm that need hessian.

enter image description here

Hence I'm not sure to understand why MAE is not sufficient since it is not differentiable only in one point and this point is when our prediction is perfect.

However my intuition is that this log cosh loss can be pretty cool near 0 to reduce parameters update by decreasing the gradient.

Am I understanding this all right? Does anyone knows any specific example where this loss is used?

Improve this question
$\endgroup$
1
  • 1
    $\begingroup$ MAE can lead to failures to converge because the optimizer continues to step over the optima provided the learning rate is large enough. $\endgroup$
    – Demetri Pananos
    May 3, 2020 at 23:07

1 Answer 1

Reset to default
1
$\begingroup$

One way to find where a topic has been deployed is a Google search. For example, if we search https://scholar.google.com/scholar?hl=en&as_sdt=0%2C47&q=log-cosh+loss&btnG=

we find a promising article

"Log Hyperbolic Cosine Loss Improves Variational Auto-Encoder" Pengfei Chen, Guangyong Chen, Shengyu Zhang. The authors claim "We propose to train VAE with a new reconstruction loss, the log hyperbolic cosine (log-cosh) loss, which can significantly improve the performance of VAE and its variants in output quality, measured by sharpness and FID score."

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.