Assuming I have loss function
f(y_pred,y_target) that I will use to train my neural network. In this case the loss function is a regression, and let's say it should work on something that is not directly solvable by something like a mean squared error loss function. An example would be an angle, which gives the rotation of an object, due to the definition of the circle**.
I am not asking for a loss function for this problem, I want to know if there is a good way to test, if your loss function is correct.
I made two assumption which must be true for a loss function for work:
- The higher the discontent with the result, the higher the loss
- The derivation of the loss function should only be zero when the loss function itself is zero
One could now test this mathematically, however I am looking for a more programmable approach, like using a stochastic input and an error with a certain distribution added and expecting the same distribution in the loss function.
So, is there a typical approach to this? Some kind of "roadmap" to test the loss function itself to guarantee, that it will do the right job?
** a simple regression will not work, because 0 and 2pi,4pi and so on will mean the same