I understood how the logistic model works and what it represents (log-odds). All the information on how the parameters are fit only evolved around the statistical way of maximizing the log-likelihood. In machine learning, at least how I understood it, the problem is usually tackled from the other side. Meaning we measure how 'wrong' the model is by a loss function. The default objective in sklearns implementation is the log-loss with l2 regularization:

$\min_{\mathbf{w},c} \frac{1}{2}\mathbf{w}^T\mathbf{w}+C\sum_{i=1}^n \ln(1+e^{-y_i(\mathbf{x_i}^T\mathbf{w}+c)}).$

I have troble understanding where the small c variable stems from and why it is minimized. Can someone please explain?

Best, Jonas


$c$ is the intercept, often denoted $\beta_0$. It is not minimized but the loss function is minimized with $\mathbf{w}$ and $c$ as parameters.

  • $\begingroup$ Yes thank you, I just found out by looking through the associated papers and how they defined the notation.I was just confused because I personally used the notation where the intercept is combined into the w. $\endgroup$ Mar 22 '20 at 10:08
  • $\begingroup$ Yes it is a bit non-standard, I guess it is to match the notation with the X input in the API, which is easier for users as it does not require adding a column of ones beforehand. $\endgroup$
    – udscbt
    Mar 22 '20 at 10:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.