Assuming I have loss function f(y_pred,y_target) that I will use to train my neural network. In this case the loss function is a regression, and let's say it should work on something that is not directly solvable by something like a mean squared error loss function. An example would be an angle, which gives the rotation of an object, due to the definition of the circle**.

I am not asking for a loss function for this problem, I want to know if there is a good way to test, if your loss function is correct.

I made two assumption which must be true for a loss function for work:

  1. The higher the discontent with the result, the higher the loss
  2. The derivation of the loss function should only be zero when the loss function itself is zero

One could now test this mathematically, however I am looking for a more programmable approach, like using a stochastic input and an error with a certain distribution added and expecting the same distribution in the loss function.

So, is there a typical approach to this? Some kind of "roadmap" to test the loss function itself to guarantee, that it will do the right job?

** a simple regression will not work, because 0 and 2pi,4pi and so on will mean the same


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.