In a binary classification problem, if both logistic regression and a single preceptron uses Sigmoid function, what's the difference in
- classification results, since they will have the same decision boundary ?
- their conceptual behaviour?
$\begingroup$
$\endgroup$
1
-
$\begingroup$ related stats.stackexchange.com/questions/137834/… $\endgroup$– Haitao DuCommented May 4, 2021 at 12:24
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
1
Their mechanics are the same, one neuron, uses linear combination of features, with intercept option and logit (sigmoid) function. Conceptually, they both try to find a hyperplane. If you also use the same loss function, they'll respond similarly.
One difference is that a typical logistic regression implementation uses Newton Conjugate Gradient solver (that uses Hessian matrix), while in neural net implementations this is not provided as an option due to its computational complexity. See sklearn implementations and solver options: logistic-regression, MLP.
-
$\begingroup$ we can use the same loss? see this discussion If you also use the same loss function, they'll respond similarly. $\endgroup$ Commented May 4, 2021 at 12:25