0
$\begingroup$

Assume I predicted the probability for samples to be in the positive class using logistic regression for binary targets. The samples come in groups of 5 but I'm predicting individually and I would like to ensure that the sum of probabilites for the positive class in one group sum up to one.

For example the following rows are the

prob pos., prob neg.]
[0.1, 0.9]
[0.4, 0.6 ]
[0.2 , 0.8 ]
[0.1 , 0.9 ]
[0.3 , 0.7 ]

so the sum for the positive label is 1.1. I'm looking for a machine learning algortihm which naturally fixes this issue or another method. Right now I'm just normalizing along the column axis but I think I loose some predictive power doing just that.

After doing some research on my own, I came across listwise learning to rank algorithms. Do they fix this? Or can I use the rank implementations in xgboost for that?

$\endgroup$
1
$\begingroup$

The softmax function is like a logistic function for multiple outputs. The theory for the softmax function shows that if you have many predicted probabilities from logistic functions then you can divide each probability by the sum of all the probabilities to get an adjusted probability. This adjustment to the probability will force them to sum to 1.

$\endgroup$
3
  • $\begingroup$ Ok, just to clarify: [0.1/1.1, 0.4/1.1, ... ] as I have it implemented is fine here; and that can be shown by considering the theory behind the softmax function? $\endgroup$ – jens0r Mar 4 '17 at 16:20
  • $\begingroup$ @jens0r this is exactly what the softmax was designed for. I mean... it's true by definition, you don't really need to show it by considering the theory. $\endgroup$ – Hugh Mar 4 '17 at 16:31
  • $\begingroup$ I get those probabilities ( 0.1, 0.4, ...) from a log reg model which uses the logit function. I find it very intuitive to just normlize in the usual way but I don't see the direct connection to the softmax $\endgroup$ – jens0r Mar 4 '17 at 19:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.