1
$\begingroup$

Suppose we have a set of experts which predict on a data set, and the true labels are also given.

I would like to find out the probabilities for combining predictions of separate experts. So the final prediction for each data instance is a randomly selected prediction of one of the experts according to the probability distribution, and this distribution should maximise the classification accuracy.

My idea was using using Theorem 4.2 (p. 127) from Kuncheva, L. I. (2004). Combining pattern classifiers: methods and algorithms, which says that the optimal combination weights for this case are $w_i=log(\frac{p_i}{1-p_i})$, where $p_i$ is the classification accuracy of the $i$-th expert. However, here the combination rule is to select the label which gets the highest cumulative weighted vote, and $w$ are not the probabilities. Then the question is, how could I convert $w$ into probabilities. I thought of projection onto probability simplex, but not quite sure if this is the best way.

Alternatively, perhaps there is a different, better way of doing this or the whole thing doesn't make any sense. Please let me know.

Thanks!

Improve this question
$\endgroup$
2
  • $\begingroup$ Seems like the softmax function would do the job. $\endgroup$
    – Sycorax ♦
    Commented May 14, 2019 at 17:41
  • $\begingroup$ It is certainly a possibility, interestingly, here is a thread discussing the relation of softmax to unit projection, where the answer states that the softmax is a crude approximation of the actual projection math.stackexchange.com/questions/2372487/… $\endgroup$
    – shiftyscales
    Commented May 15, 2019 at 16:24

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.