I have looked at the logistic loss function at many different sources, and many places I find it plotted like shown here:

enter image description here

Taken from http://fa.bianp.net/blog/2013/loss-functions-for-ordinal-regression/

The logistic loss is typically defined as $L(y_i, f(x_i)) = log(1+exp(x_i y_i))$ where $y_i = \pm \;1$.

What confuses me is that when calculating the logistic loss for $x_i = 0$, I cannot see how to get a value of $1$ when inserting into the formula:

$L(1, f(0)) = L(-1, f(0)) = log(1+exp(0)) \approx 0.69$

Have I misunderstood the concept? I can only see that a loss of $1$ can be returned by using a logarithm with base $1$, but I cannot imagine that would be anyone's intention.


A logarithm with base 2 would give $log_2(1 + exp(0)) = log_2(1+1) = log_2(2) = 1$.

Logarithms aren't defined for base 1; you can see how it isn't really convenient to define $log_1$ by considering the formula for change of base: $log_b(x) = \frac{log_k(x)}{log_k(b)}$. If we were trying for $log_1$ we'd get $log_k(1) = 0$ in the denominator, which is inconvenient. Or consider that we're trying to find a number $p$ so that $1^p = x$, which is only possible for $x=1$, and then you could use any $p$. See this Q and also this Q on the math forums.

| cite | improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.