1
$\begingroup$

I have a 3 class problem for which the outcomes can be described below using a confusion matrix. 1 are the outcomes I want the model to maximize and 5 are the outcomes I want the model to minimize. In other words, if the model is going to predict correctly, squares with 1 are more important than squares with 2. If the model is going to predict incorrectly, it better not predict square 5 incorrectly.

enter image description here

Are there any general guidelines I should follow or things to keep in mind for creating my own cost function or can I just play around until I make one that fits what I'm looking for. I would imagine one requirement is differentiable. Anything else?

Improve this question
$\endgroup$
3
  • 1
    $\begingroup$ What should be differentiable? Your cases and outcomes seem to be discrete. You could just multiply the cases by the values in the table and add the results up $\endgroup$
    – Henry
    Jun 19, 2020 at 13:30
  • $\begingroup$ The cost function/loss function that you are looking to minimize or maximize - I thought would need to be differentiable. So there really aren't many rules of thumbs for creating your own cost function? It's more trial/error based. One can treat the form of the cost function as an additional hyperparameter that needs to be tweaked? $\endgroup$
    – confused
    Jun 21, 2020 at 15:16
  • 1
    $\begingroup$ Related: "Evaluating classification results when importance of correct classification varies with class". $\endgroup$
    – Richard Hardy
    Jun 21, 2020 at 15:37

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.