Is classification using linear regression called logistic regression or linear disriminant analysis?

Question

I have heard people describe logistic regression as linear regression except as it is deployed for classification. But I have heard the exact same comment about LDA (linear discriminant analysis). Out of logistic regression and LDA, which is closer to what happens in linear regression?

Answer 1

They are both close, but in different ways

• If you run ordinary least-squares regression with a binary class variable as the outcome (label) variable, you get exactly the 2-class case of linear discriminant analysis. So LDA (in the 2-class case) is linear regression run on a classification problem. It's conceptually different from linear regression in that the original derivation of LDA uses assumptions about the distribution of the predictor (feature) variables, which regression does not.
• Logistic regression is a natural generalisation of linear regression to binary data, in which you model the mean of the outcome variable (which is the probability that it is 1 vs 0) using a linear combination of predictors, but with a 'link' function in between so that the probability stays between 0 and 1. Like linear regression, it's a special case of the generalised linear model, and wasn't derived based on assumptions about the distributions of predictor variables.

So, LDA is computationally just linear regression as applied to a classification problem, but it's quite different as a model; logistic regression is closer as a model, but less similar computationally.