I have a 3 class problem for which the outcomes can be described below using a confusion matrix. 1 are the outcomes I want the model to maximize and 5 are the outcomes I want the model to minimize. In other words, if the model is going to predict correctly, squares with 1 are more important than squares with 2. If the model is going to predict incorrectly, it better not predict square 5 incorrectly.
Are there any general guidelines I should follow or things to keep in mind for creating my own cost function or can I just play around until I make one that fits what I'm looking for. I would imagine one requirement is differentiable. Anything else?