I am dealing with a problem where the output of my model, can have numbers like 1-3000 (around) (score in a game). This is like a score in a game. Giving a least squared error setting, for a model, where I want to try to predict the score, will make me end up with a large error, and will not be quite reflective of a good prediction. For example, for a game of score 2800, I might be happy with a prediction of 2100, but for a game with score 300, I definitely will not be happy with a prediction of 1000, even though both have the same offset of 700.
At the same time, if I have a classification like prediction over all the 3000 classes (let's define each discrete score in the range to be a class) and then give my results as a discrete probability distribution over the scores, then I am not using the closeness of scores in the picture, because the model would assume all classes are independent, but a score of 1900 is more close and similar to a class of score 2000 than a class of score 1.
So, how exactly should I come up with a method to learn and evaluate , with a good interpretable ground truth when I am trying to build a model to predict such data (like game scores)?
Thanks in advance for all help.