I have a 3 class problem for which the outcomes can be described below using a confusion matrix. 1 are the outcomes I want the model to maximize and 5 are the outcomes I want the model to minimize. In other words, if the model is going to predict correctly, squares with 1 are more important than squares with 2. If the model is going to predict incorrectly, it better not predict square 5 incorrectly.

enter image description here

Are there any general guidelines I should follow or things to keep in mind for creating my own cost function or can I just play around until I make one that fits what I'm looking for. I would imagine one requirement is differentiable. Anything else?

  • 1
    $\begingroup$ What should be differentiable? Your cases and outcomes seem to be discrete. You could just multiply the cases by the values in the table and add the results up $\endgroup$
    – Henry
    Jun 19, 2020 at 13:30
  • $\begingroup$ The cost function/loss function that you are looking to minimize or maximize - I thought would need to be differentiable. So there really aren't many rules of thumbs for creating your own cost function? It's more trial/error based. One can treat the form of the cost function as an additional hyperparameter that needs to be tweaked? $\endgroup$
    – confused
    Jun 21, 2020 at 15:16
  • 1


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.