Proposed Solution: Calibration
I have read the "Multi-class" part of your post, but since you are using One vs. All SVMs, i think you should reconsider solving the problem at the binary level. You could calibrate the single svms, so that the resulting output values are comparable.
Calibration methods for the binary SVMs (so that means also applicable in the one vs. all scenario) are Platt scaling1 and Isontonic regression. A nice overview with python code is available here.
For your own use case you would then calibrate each OvA SVM separately and afterwards the calibrated outputs for
c should be comparable.
What does calibration do here?
The key thing here is, that SVMs themselves, are not probabilistic. The output value you mentioned is usually a function of the classified point's distance to the hyperplane. So we are using a heuristic which has no further significance. The goal of this heuristic is that higher numbers are more likely to be the correct result.
You can measure the signifance of your output values using a
reliability plot. I will cut short here but essentially you want your reliability curve to be as close as possible to the diagonal. The calibration adds another mapping of output values to calibrated output values. This can handle for example classifiers which have a bias towards high output values. Think of it as another translation step "Ok i got that really confident
0.9 from you classifier A, but i know you always are over-confident so let's make this a
0.5". So a
0.5 value of classifier A should be closer to a
0.5 value of classifier B in the end.
Keep in mind, when using calibration you have to work thoroughly as usual (train/dev/test set).
1. Platt, J. (1999). Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods. Advances in large margin classifiers, 10(3), 61-74.