1
$\begingroup$

I have a score of some feature, $F_1$ and $F_2$ where this score is the logarithm of probability.

This score is very low and very sparse, for example i have:

$F_1$ with score $-800$

$F_2$ with score $-3000$

I need to get the probability with the exponential function and make the sum of probability like this:

$e^{-800} + e^{-3000}$

But if I make the exponential of this number i get 0 for the underflow problem. With log-sum-exp trick i can do:

$\log(e^{-800}+e^{-3000})$

$= \log(e^{-800+800}+e^{-3000+800})-800$

$= \log(e^{0}+e^{-2200})-800$

But now i have $e^{-2200}$ that is too small.

What's wrong? how I can make my $e^{-800} + e^{-3000}$ sum ?

Improve this question
$\endgroup$
4
  • 5
    $\begingroup$ Please explain why underflow is any problem at all. To a very high precision, it is indeed the case that $\exp(-800) + \exp(-3000) = \exp(-800)$, as your analysis shows. Do you really need $955+$ decimal digits of precision in your numbers? $\endgroup$
    – whuber ♦
    Jul 9, 2014 at 16:20
  • $\begingroup$ Yes because my probability is like 1E-95, if i decrease the precision i get 0. $\endgroup$
    – Neptune
    Jul 9, 2014 at 16:28
  • 3
    $\begingroup$ I don't think you follow my question: you are asking how to compute $\log(1 + \exp(-2200))$. This value is approximately $10^{-955}$. Do you need the kind of precision that distinguishes this from $0$? If so, why? $\endgroup$
    – whuber ♦
    Jul 9, 2014 at 17:15
  • 1
    $\begingroup$ You are right, I need to search the value with maximum probability, then I can approximate to zero all the probability away from the maximum and take only the value near the maximum. $\endgroup$
    – Neptune
    Jul 10, 2014 at 18:19

0

Reset to default

Browse other questions tagged or ask your own question.