Range of arguments in loss function of logistic regression

Question

We have the following loss function for logistic regression, the so called log-loss defined as:

$- \Big[\sum_i y^{i}\log(h(x^i))+(1-y^i)\log(1-h(x^i))\Big]$

We also know that logistic regression assigns a datasample to class y=1 if the posterior probability $h(x)$ of class $y=1$ is bigger than 0.5.

Now my question: The term $y^{i}\log(h(x^i))$ quantifies the case where the true label is "$y=1$", but the prediction is "$y=0$". The prediction "$y=0$" is however only done when $h(x)<0.5$. Does this mean that the $h(x^i)$ in $y^{i}\log(h(x^i))$ always will be $<0.5$?

Answer 1

We also know that logistic regression assigns a datasample to class y=1 if the posterior probability p of class y=1 is bigger than 0.5.

This is not true; logistic regression is not a classifier. But the notation here is a little confusing because $p$ does not appear in your expression for the loss.

The term $y^{i}\log(h(x^i))$ quantifies the case where the true label is "$y=1$", but the prediction is "$y=0$".

This is not true. The way to think about the log-loss function is that $y^i$ works as a "switch." If $y^i=1$, then the term $\log(h(x^i))$ is added to the loss; if $y^i=0$, the term $\log(1-h(x^i))$ is added to the loss.

The prediction "$y=0$" is however only done when $h(x)<0.5$.

When we're considering the log-loss, at no point do we consider whether or not $h(x^i)>0.5$. If $y^i=1$ but $h(x^i) < 0.5$, even if $h(x^i)$ is very small, the expression is evaluated as it is written. Very small values $h(x^i)$ naturally imply that the loss contribution for the sample will be very large, which is exactly what we want because in that case, the model poorly predicts the sample.