How to deal with 'near miss' labels in categorization?

Question

I describe the general problem first, and then add an example for context:

I'm working on a binary classifier using supervised learning. I have a set of examples with corresponding features and each is labelled 0 or 1. So far, the classic problem of making a binary classifier . Also not unusual is that my output is noisy: whether an example is classified 0 or 1 is partly determined by the features and partly by a latent 'noise' function.
What makes this problem different is that I have a separate set of examples labelled 'near miss'. It is assumed that these examples have an above averages chance to be 1 based on their features and a portion would have been categorized as 1, if it weren't for (exceptional) 'bad luck' with regards to the noise function. So on the one hand I want a function that has as few as possible false positives (and false negatives), but on the other hand I prefer the false positives it does make to be examples in the 'near miss' category.
If the labels were numerical probabilities that the examples are 1, I could have used regression. If I wanted to separately predict which input would end up as a 'near miss' I could use categorization with 3 categories. However now it seems to me I can't really use either.

Context:
Consider the question: 'what kind of people will buy a given shirt'? You can approach this problem by getting data about a set of people and label them by whether they bought the shirt or not. However consider the set of people that spent more than 1 minute looking at the shirt. Some of those people might be looking at the shirt fascinated by how ugly it is and would never buy it. However a significant portion of people might be seriously considering the shirt and for what ever reason not go through with the purchase. Chances are that if we desperately try to exclude everyone of this group, we will overfit our function. However pretending everyone in this group has bought the shirt is nonsense. The problem is to discourage the algorithm to have false positives while at the same time to encourage it to have the false positive it does make fall into this 'near miss' category.

Answer 1

did you consider ordinal regression? There are some R packages able to deal with (e.g. rpartScore, mgcv ...), although it is not fully developed. Roughly, it looks like you have an odination with different class amplitude (e.g. '0' if 0 to 0.5 'almost 1' if 0.5 to 0.75 and '1' if 0.75 to 1). Check here: https://github.com/ayrna/orca it can give you some ideas, although it is mainly written in Matlab. Hope it helps. Rafa.