We have trained about 200 Linear Support Vector Machines (hand-coded in C#) belonging to our 200 categories and we use them in text categorization. Due to the shortness of the text in our training samples and other uncontrollable factors, very often every machine returns a negative number for (classifies as negative) a given document. Returning "None" as an answer is not feasible because it happens maybe about 40% of the time, and not classifying 40% of the documents is unacceptable from a business point of view.
What we do is, we don't bother about the sign of the results and return the category with the maximum value anyway. However, I feel this makes our error rate larger than need to be.
So here is my question: Is there a principled middle-of-the-road way? That is it will sometimes (but not too often, maybe about 5% of the time) return "None" and it will sometimes return the maximum of the negative numbers, so that |misclassified documents| + k.|unclassified documents| will be minimum (for some k<=1)? For example, is there a way to translate the distance given by an SVM to a probability value? Or, relatedly, is there a principled way to compare the output of two Support Vector Machines, i.e. +0.2 denotes a higher probability of membership in one of them? Any suggestion, source, paper is welcome. Thank you.