5
$\begingroup$

This is the description of logistic regression and probit regression as generalized linear model from wiki:

Both the logistic and normal distributions are symmetric with a basic unimodal, "bell curve" shape. The only difference is that the logistic distribution has somewhat heavier tails, which means that it is less sensitive to outlying data (and hence somewhat more robust to model mis-specifications or erroneous data).

I don't understand the conclusion somewhat heavier tails, which means that it is less sensitive to outlying data. Heavier tails means the relatively high probability of extreme values (outliers). Why do we say it is less sensitive to the extreme values?

$\endgroup$
5
  • 1
    $\begingroup$ A data point is not an outlier if it is "expected", in some sense, by the model. If you fix the data values, any extreme values in the data are more likely when viewed through the lens of a logistic distribution than when viewed through the lens of a normal distribution, therefore, the parameter estimates don't get moved around as much to try to "explain" (fit) the extreme values. $\endgroup$
    – jbowman
    Commented May 29, 2022 at 16:55
  • $\begingroup$ You have data, and you are trying to fit a model to it. It's not about generating extreme values from either a logistic or a normal, it's about trying to fit a logistic or a normal to pre-existing data that may have extreme values. The statement above has nothing to do with how the data was generated. $\endgroup$
    – jbowman
    Commented May 29, 2022 at 17:00
  • $\begingroup$ get it, if the distribution can hardly generate the extreme values, as a result, model has to move parameters largely to fit those extreme value samples during training. $\endgroup$ Commented May 29, 2022 at 17:02
  • $\begingroup$ Exactly right!! $\endgroup$
    – jbowman
    Commented May 29, 2022 at 17:03
  • $\begingroup$ @jbowman actually you can write as an answer. I think this explanation is very useful. $\endgroup$ Commented May 29, 2022 at 17:03

1 Answer 1

9
$\begingroup$

The statement is not about generating extreme values from either a logistic or a normal, it's about trying to fit a logistic or a Normal to pre-existing data that may have extreme values.

You have data, and you are trying to fit a model to it. However, a data point is not an outlier if it is "expected", in some sense, by the model. Any extreme values in the data are more likely when viewed through the lens of a logistic distribution than when viewed through the lens of a Normal distribution, therefore, the parameter estimates don't get moved around as much to try to "explain" (fit) them. This can be restated as the parameter estimates being less sensitive to outliers when we use a logistic distribution than when we use a Normal distribution.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.