7
$\begingroup$

Logistic regression, used in binary classification, uses the logistic function as a model for the underlying probability of the outcome variable.

It has some properties useful and essential for fitting such a model. For example it is monotonically increasing, it tends to 1 as x tends to infinity, it tends to 0 as x tends to minus infinity, it is never 0 nor 1 (allowing for positive probability of either outcome regardless of input). However, there are other options for function which satisfies these properties.

So is the logistic function used simply for convenience, or are there other motivations for why logistic function is the "correct" or only suitable function to use?

$\endgroup$
9
$\begingroup$

There are several reasons for choosing the logistic function as a "default" method for estimating probabilities from one or more variables. Here are a few:

  1. Historical, e.g. dose-response curves
  2. When used with a regression specification on the right hand side of a model, the regression effects are interpretable in that they can be related to odds ratios for the separate effects of predictors
  3. If you start with a multivariate normality assumption for the predictors as in linear discriminant analysis, using Bayes' rule to reverse the conditioning yields the logistic model
  4. The shape fits actual data a good deal of the time

Please note that the logistic is not used for classification but for direct probability estimation.

$\endgroup$
2
$\begingroup$

A technical feature of the logistic link function is that when combined with ML estimation (or posterior mode with flat prior) the gradient equation becomes

$$\sum_i x_i (p_i - y_i) = 0 $$

That is, your residuals, $p_i - y_i$, are uncorrelated with the covariates, $x_i$ (note: $y_i $ is binary $0$ or $1$). This is analogous to ols regression. If you have a different link function - the equation gets modified by inclusion of "weights" that depend on how different the link function it is from logistic one. One practical feature of this is that if you include an intercept in your model, the fitted probabilities will add up to the number of "successes" (observations where $y_i=1$). Similarly for factor variables.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.