2
$\begingroup$

I've always assumed that logistic regression is called "logistic" because the model directly uses the logistic function.

However, these Stanford notes seem to imply that the name comes from the logistic loss:

The different loss functions lead to different machine learning procedures; in particular, the logistic loss ϕlogistic is logistic regression, the hinge loss ϕhinge gives rise to so-called support vector machines, ...

This doesn't seem quite correct to me, since

  1. You could use logistic loss with a different model, e.g. a neural network
  2. You could use a different loss with logistic regression, e.g. hinge loss

Or am I missing something?

Improve this question
$\endgroup$
1

1 Answer 1

Reset to default
2
$\begingroup$

Logistic regression is a name for the whole model, i.e. linear estimator, logit link function (hence, logistic function is an inverse of it), and Bernoulli distribution as likelihood (maximizing it is equivalent of minimizing logistic loss). You need all the elements for it to be proper logistic regression.

The above is standard definition is statistics, in machine learning the name is used in less formal fashion. Logistic loss needs bounded outputs, so you cannot use it without logistic function for the model. In machine learning, by default you just minimize loss, so the model may be changed to other model by switching the loss (e.g. to hinge). Moreover, in machine learning, the model may have other modifications, e.g. regularization, that are not a part of the standard statistical model.

See also the What does the name "Logistic Regression" mean? thread.

Improve this answer
$\endgroup$
5
  • $\begingroup$ Is LOGit link function, LOGistic loss names somehow connected with logarithm ? $\endgroup$
    – Brans
    Aug 26, 2020 at 9:24
  • $\begingroup$ @Brans see the link I added in edit. $\endgroup$
    – Tim ♦
    Aug 26, 2020 at 9:28
  • 2
    $\begingroup$ @Brans Yes, as both logit and logistic are related to the logarithm of odds. Which makes the peculiar pronunciation of the o by economists incomprehensible (a similar thing happens with probit which they make sound more like probe than probability) $\endgroup$
    – Henry
    Aug 26, 2020 at 9:45
  • 1
    $\begingroup$ Does anyone say prob-it? I only ever hear pro-bit, from biostatisticians too. I owe @Henry the insight that the pronunciation is at odds with the etymology. Logit was a coinage by Berkson in 1944, IIRC, but log odds appeared naturally in statistics some time before. Logistic curves for population and other growth curves go back to Verhulst in the 19th century and were in a sense pre-statistical, as "Look! it fits (except in so far as it doesn't)" was often the prevailing style. Pure mathematicians might want to stress the kinship with Riccati equations. To all comments add "if not earlier". $\endgroup$
    – Nick Cox
    Aug 26, 2020 at 10:12
  • 1
    $\begingroup$ @Tim Makes sense. I assume that the quote about logistic loss "giving rise" to logistic regression is wrong/misleading. $\endgroup$
    – kennysong
    Aug 27, 2020 at 1:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.