I am building a classification model, on whether a particular outcome occurs or not. For each observation, there is an associated weight which is unique by observation, and should penalize misclassification of that observation.
Are there any classification metrics that lend themselves well to per-observation weighted errors?
I've seen cases where we can apply weights globally to a particular type of misclassification, i.e. TP or FN, but I've been unable to find much in the way of weighing specific observations differently.
The more I think about it, the more it starts to look like a regression, however, the output from my model needs to be a probability bounded between 0 and 1.