Since Logistic Regression is a statistical classification model dealing with categorical dependent variables, why isn't it called Logistic Classification? Shouldn't the "Regression" name be reserved to models dealing with continuous dependent variables?

  • 5
    Logistic regression belongs to the GLM family of models. – Stéphane Laurent Dec 7 '14 at 19:04
  • 10
    You can use it to regress probabilities. – Emre Dec 7 '14 at 22:38
  • 21
    While logistic regression can certainly be used for classification by introducing a threshold on the probabilities it returns, that's hardly its only use - or even its primary use. It was developed for - and continues to be used for - regression purposes that have nothing to do with classification. I'd argue that this is still easily what it's mostly used for, but I suppose it depends on what you look at. – Glen_b Dec 8 '14 at 1:07
  • 6
    You might find this paper on the development of logistic regression interesting, particularly since it does give some sense of the kinds of problems that it is used for as a regression technique. – Glen_b Dec 8 '14 at 1:14
up vote 92 down vote accepted

Logistic regression is emphatically not a classification algorithm on its own. It is only a classification algorithm in combination with a decision rule that makes dichotomous the predicted probabilities of the outcome. Logistic regression is a regression model because it estimates the probability of class membership as a (transformation of a) multilinear function of the features.

Frank Harrell has posted a number of answers on this website enumerating the pitfalls of regarding logistic regression as a classification algorithm. Among them:

If I recall correctly, he once pointed me to his book on regression strategies for more elaboration on these (and more!) points, but I can't seem to find that particular post.

  • That makes sense. Would you have a link to one of these articles? – Ismael Ghalimi Dec 7 '14 at 18:52
  • If that's the case, all(or most) the classifiers predicts the probabilities to belong in a class first(as far as I know) and then transform this prob to classes.. Don't they? – Outlier Dec 10 '14 at 6:51
  • 4
    @Outlier Counterexample: SVM doesn't compute class probabilities at all, it just measures the distance between an observation and a hyperplane. – Sycorax Dec 10 '14 at 12:59

Abstractly, regression is the problem of calculating a conditional expectation $E[Y|X=x]$. The form taken by this expectation is different depending on the assumptions of how the data were generated:

  • Assuming (Y|X=x) to be normally distributed yields with classical linear regression.
  • Assuming a Poisson distribution yields Poisson regression.
  • Assuming a Bernoulli distribution yields logistic regression.

The term "regression" has also been used more generally than this, including approaches like quantile regression, which estimates a given quantile of $(Y|X=x)$.

Your Answer

 
discard

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.