# Does Linear regression needs target variable to be normally distributed. (GLM context)?

I came across the assumptions of linear regression that said: -->The residuals should be normally distributed.

GLM(Generalized Linear model) assumes that target variable should follow one of the exponential family.

So does linear regression needs residuals as well as target variable to be distributed normally?

EDIT

https://online.stat.psu.edu/stat504/node/216/

In the above mentioned, it is written -

There are three components to any GLM:

1. Random Component – refers to the probability distribution of the response variable (Y); e.g. normal distribution for Y in the linear regression, or binomial distribution for Y in the binary logistic regression.

Moreover in the assumption section,

The dependent variable Yi does NOT need to be normally distributed, but it typically assumes a distribution from an exponential family (e.g. binomial, Poisson, multinomial, normal,...)

I am new to machine learning, forgive me if i'm asking stupid question.

• Answered several times on site already. No statistical procedure assumes the raw (unconditional) response variable has any specific distribution either for regression or for a GLM. I.e. looking at histograms, QQ plots or tests on the response itself is pointless. The assumption -for tests, CI and prediction intervals relates only to conditional distributions. Aug 26, 2020 at 4:52
• I have edited my post, please have a look. The link that i have mentioned mentions that for linear regression, target variable should be normally distributed.(Also mentioned in question). Aug 27, 2020 at 10:02
• Yes, I can point to more things like that, but those quotes are misleading you. Note the presence of $x_i$ (the vector of predictor values for obs. $i$, in the notation of your link) in the expression for the linear predictor (and hence the mean). This means that $y_i|{x}_i$ typically has a different distribution-parameter from $y_j|{x}_j$ (unless their x-vectors are identical), and combining them all (by looking at the marginal distribution) you end up with a mixture of different distributions (same family but different parameters), which is not the same as the conditional distribution... Aug 28, 2020 at 2:01
• ... while $y_i|{x}_i$ may be Gaussian (or Gamma or Poisson or binomial etc depending on the GLM), the collection of random variables across different $x$-vectors is not Gaussian (or Gamma or Poisson etc); it depends on the distribution of the $x$'s. Again, I encourage you to try some searches, since this is a pretty well-worn topic on this site. Aug 28, 2020 at 2:05

It depends on what you’re doing. If you just want to predict, then it doesn’t matter, and the Gauss-Markov theorem does not say anything about a normal error term.

However, when the error term is normal, then the OLS estimator $$\hat{\beta}$$ is the maximum likelihood estimator. If you don’t know about MLEs, you’ll see them over and over as you dive into statistics, but maximum likelihood is a nice property for many reasons.

Among those reasons is that the inferential methods like p-values on coefficients and F-tests of nested models come into play.

So if you want to do some kind of ANOVA, for example, the normality of the error term matters because you’re doing hypothesis testing, not prediction.

The pooled distribution of the response variable (all of your $$y$$s) definitely does not have to be normal, even to get that maximum likelihood property and do inference, and the predictor variables definitely don’t have to be normal. Predictors often cannot be normal, such as when they are categorical variables e.g. male/female, treatment/control, etc.

EDIT

We often talk about normal residuals. This is casual language, and experienced statisticians know what is meant, but the residuals are a discrete distribution and cannot be normal. What we assume is a normal error term, and we use the residuals to gauge if that is a good assumption or not.

• (+1) residuals are a discrete distribution and cannot be normal I'm confused by this... My interpretation is that residuals are realizations (draws) from what is assumed to be a normal distribution. As such, they are unlikely to match exactly the theoretical distribution. Is this what you mean by "discrete distribution" - am I getting this wrong? Feb 15, 2022 at 12:41
• @dariober You got it! I’m totally content to use the slang with experienced people, but I do think it’s important to clarify to newcomers what’s going on. Learn it right and then take linguistic shortcuts.
– Dave
Feb 15, 2022 at 12:46

The only normality assumption in linear regression if you intend to do any testing is that the residuals be normally distributed. In simple linear regression with only one variable in the model, this implies that the independent variable must also be normally distributed. In multiple regression, however, the residuals must be normally distributed.

• This is incorrect. The Gauss-Markov theorem does not require normality.
– Dave
Aug 26, 2020 at 3:41
• The question wasn't about Gauss-Markov theorem. The question is about regression and all that it entails. Aug 26, 2020 at 3:43