1
$\begingroup$

I am reading Statistical Rethinking (Section 4.2).

When defining the components of a model description the author says:

... we define a likelihood distribution that defines the plausibility of individual observations. In linear regression, this distribution is always Gaussian.

Why is the likelihood distribution always Gaussian for linear regression?

Edit:

After re-reading the chapter introduction, the author states:

This chapter introduces linear regression as a Bayesian procedure. Under a probability interpretation, which is necessary for Bayesian work, linear regression uses a Gaussian (normal) distribution to describe our golem’s uncertainty about some measurement of interest. This type of model is simple, flexible, and commonplace.

which specifies the context for the use of the Gaussian distribution in this case.

$\endgroup$
2
  • 2
    $\begingroup$ The distribution isn't alway Gaussian in linear regression. The author is only talking about what he does in the book. $\endgroup$
    – Olivier
    Feb 13, 2018 at 3:06
  • 1
    $\begingroup$ related: stats.stackexchange.com/questions/29731/… $\endgroup$
    – Taylor
    Feb 13, 2018 at 3:07

1 Answer 1

1
$\begingroup$

enter image description here

This image gives a good probabilistic interpretation for Linear regression. We want the mean of the residual (fancy word for error) at every point on our line to be zero. We also want a good majority of the points to be close to the mean (our prediction value) - what this means is that we don't want our variance to be too high. When we maximize the likelihood for each point with respect to the normal distribution we can clearly see that the max likelihood would occur if all points were on the linear predictor (the mean). If we find points that are slightly further away from the mean, from either direction, we would penalize the likelihood relative to how far the data points are.

With this in mind, we can see why a normal distribution would be pretty convenient. Firstly, we often don't want to penalize negative errors more than positive errors, so a symmetric distribution is called for. Secondly, Normal distributions occur very frequently in datasets, so it is a really natural choice.

Lastly, the wording of the textbook isn't exactly accurate, because you could technically use a different distribution if you wanted to. The t-distribution is also most notably used in special circumstances instead of a normal one.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.