# How to do one-vs-one classification for logistic regression?

I have a dataset with 4 clases and I want to apply logistic regression with one-vs-one classification. So, first I train for each pair of classes a logistic regression classifier (i.e. calculate the weights (theta)). Then, for a new data point I apply all the previously trained classifiers and take the class with the most votes as the winning class.

But how should I handle ties, i.e. when two or more classes have the same amount of votes? Should I just pick the class with the heighest weight (theta) among these classes or should I sum up the weights (theta) for these classes and then pick the class with the highest value?

• You are using an improper and discontinuous accuracy score, i.e., a score that is optimized by choosing the wrong features and estimating their importance using the wrong weights. – Frank Harrell Dec 17 '15 at 12:52

Each hypothesis function that outputs the result of logistic regression ($$h_\theta (x)$$) corresponds to the probability that the inputted feature vector $$\vec{x}$$ is either a $$1$$ or a $$0$$ (where 1 is the 'yes'/'true'/etc. term.)
So now that you realize that each model you're creating outputs the probability that you want to know what a 'tie' means. If 2 models both output 0.4 as the probability that the inputted feature vector is part of that class, you could say that between the two the answer is indeterminate, or rather that input $$\vec{x}$$ is equally likely to be in 'class 1' or 'class 2' (assuming $$P(y=1|x;\theta)=P(y=2|x;\theta)=\phi$$ where $$\phi$$ is some number, which happened to be 0.4 in this example)