0
$\begingroup$

I've always assumed that logistic regression is called "logistic" because the model directly uses the logistic function.

However, these Stanford notes seem to imply that the name comes from the logistic loss:

The different loss functions lead to different machine learning procedures; in particular, the logistic loss ϕlogistic is logistic regression, the hinge loss ϕhinge gives rise to so-called support vector machines, ...

This doesn't seem quite correct to me, since

  1. You could use logistic loss with a different model, e.g. a neural network
  2. You could use a different loss with logistic regression, e.g. hinge loss

Or am I missing something?

$\endgroup$
2
$\begingroup$

Logistic regression is a name for the whole model, i.e. linear estimator, logit link function (hence, logistic function is an inverse of it), and Bernoulli distribution as likelihood (maximizing it is equivalent of minimizing logistic loss). You need all the elements for it to be proper logistic regression.

The above is standard definition is statistics, in machine learning the name is used in less formal fashion. Logistic loss needs bounded outputs, so you cannot use it without logistic function for the model. In machine learning, by default you just minimize loss, so the model may be changed to other model by switching the loss (e.g. to hinge). Moreover, in machine learning, the model may have other modifications, e.g. regularization, that are not a part of the standard statistical model.

See also the What does the name "Logistic Regression" mean? thread.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ Is LOGit link function, LOGistic loss names somehow connected with logarithm ? $\endgroup$ – Brans Aug 26 at 9:24
  • $\begingroup$ @Brans see the link I added in edit. $\endgroup$ – Tim Aug 26 at 9:28
  • 2
    $\begingroup$ @Brans Yes, as both logit and logistic are related to the logarithm of odds. Which makes the peculiar pronunciation of the o by economists incomprehensible (a similar thing happens with probit which they make sound more like probe than probability) $\endgroup$ – Henry Aug 26 at 9:45
  • $\begingroup$ Does anyone say prob-it? I only ever hear pro-bit, from biostatisticians too. I owe @Henry the insight that the pronunciation is at odds with the etymology. Logit was a coinage by Berkson in 1944, IIRC, but log odds appeared naturally in statistics some time before. Logistic curves for population and other growth curves go back to Verhulst in the 19th century and were in a sense pre-statistical, as "Look! it fits (except in so far as it doesn't)" was often the prevailing style. Pure mathematicians might want to stress the kinship with Riccati equations. To all comments add "if not earlier". $\endgroup$ – Nick Cox Aug 26 at 10:12
  • $\begingroup$ @Tim Makes sense. I assume that the quote about logistic loss "giving rise" to logistic regression is wrong/misleading. $\endgroup$ – kennysong Aug 27 at 1:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.