The decision boundary of SVM is a straight line. If we use e.g. RBF kernel, decision boundary is linear in hilbert space, but it the original space it is non-linear. I assume that the logistic regression has linear decision boundary. Is it linear also after using of penalty term, such as elastic net?
$\begingroup$
$\endgroup$
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
2
The penalty terms are regularisation terms that affect the training process and thus the decision boundary found (i.e. weights and/or bias). Once trained, the prediction is not affected by neither the regularisation method nor the loss function. So, any linear classifier/regressor is still linear after penalisation. That said, since logistic regression classifier has a linear decision boundary, it'll still have a linear decision boundary when penalised.
-
$\begingroup$ Logistic regression is actually a non-linear model. It is classified as a general linear model, but like most general linear models they are not actually linear $\endgroup$ Commented Mar 7, 2020 at 23:47
-
1$\begingroup$ @strateeg32 I said logistic regression classifier in my post, not logistic regression. It's used with a threshold which boils down to a classifier in the form $w^Tx+b>\tau$, a hyper-plane. $\endgroup$– gunesCommented Mar 7, 2020 at 23:52