I've done a classification with different machine learning method, as result i've the confusion matrix and the roc curve. To find best classification model, i'm looking for a method to combine the indicators computed with the confusion matrix, for example i've thought :
score=efficiency+purity+completeness-contamination
or
score=(efficiency+purity+completeness)/contamination
and then find the model with the max score. What is the better criterion?
Let's say you train a neural network using cross-entropy to classify inputs into one of four categories. Let's say live input x_i yields:
[0 0.45 0.55 0]
Let's also say you train a Naive Bayes classifier to categorize inputs, and for the same x_i input above, it gives you:
[0.2 0.3 0.1 0.4]
Averaging the results for the Neural Network and Naive Bayes classifiers, we get:
[((0+0.2)/2) ((0.45+0.3)/2) ((0.55+0.1)/2) ((0+0.4)/2)]
Which yields:
[0.1 0.375 0.325 0.2]
Using this ensemble approach, we classify the input x_i in the second category, whereas each method on their own predicts a different category.
