1
$\begingroup$

The decision boundary of SVM is a straight line. If we use e.g. RBF kernel, decision boundary is linear in hilbert space, but it the original space it is non-linear. I assume that the logistic regression has linear decision boundary. Is it linear also after using of penalty term, such as elastic net?

$\endgroup$
1
$\begingroup$

The penalty terms are regularisation terms that affect the training process and thus the decision boundary found (i.e. weights and/or bias). Once trained, the prediction is not affected by neither the regularisation method nor the loss function. So, any linear classifier/regressor is still linear after penalisation. That said, since logistic regression classifier has a linear decision boundary, it'll still have a linear decision boundary when penalised.

$\endgroup$
2
  • $\begingroup$ Logistic regression is actually a non-linear model. It is classified as a general linear model, but like most general linear models they are not actually linear $\endgroup$
    – strateeg32
    Mar 7 '20 at 23:47
  • 1
    $\begingroup$ @strateeg32 I said logistic regression classifier in my post, not logistic regression. It's used with a threshold which boils down to a classifier in the form $w^Tx+b>\tau$, a hyper-plane. $\endgroup$
    – gunes
    Mar 7 '20 at 23:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.