For logistic regression, the loss function is convex or not? Andrew Ng of Coursera said it is convex but in NPTEL it is said is said it is non convex because there is no unique solution. (many possible classifying line)
$\begingroup$
$\endgroup$
1
-
4$\begingroup$ The convexity of logistic regression is demonstrated in What is happening here, when I use squared loss in logistic regression setting? $\endgroup$– Sycorax ♦Commented Jun 13, 2019 at 0:38
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
I don't think anybody claimed that it isn't convex, since it is convex (maybe they meant logistic function or neural networks). Let's check 1D version for simplicity
$$L = - t \log(p) + (1 - t) \log(1-p)$$
Where $p = \frac{1}{1 + \exp(-wx)}$
$t$ is target, $x$ is input, and $w$ denotes weights.
L is twice differentiable with respect to $w$ and $\frac{d}{dw^2} L = \frac{x^2 \exp(wx)}{(1 + \exp(wx))^2} > 0$, so the loss function is convex.
answered Jun 12, 2019 at 12:29