What are the myths associated with linear regression, data transformations? I have been encountering many assumptions associated with linear regression (especially ordinary least squares regression) which are untrue or unnecessary. For example:

*

*independent variables must have a Gaussian distribution

*outliers are the points either above or below the upper or lower whiskers correspondingly (employing Boxplot terminology)

*and that the sole purpose of transformations is to bring a distribution close to normal in order to suit the model.

I would like to know what are the myths that are generally taken for facts/assumptions about linear regression, especially concerning associated nonlinear transformations and distributional assumptions.  How did such myths come about?
 A: Where do these ideas come from?
Poor texts (correction: very poor texts) after treating descriptive statistics often include some more or less mangled version of an idea that (1) you ideally need normally distributed variables to do anything inferential, or else (2) you need non-parametric tests. Then they may or may not mention that transformations could get you nearer (1).
The first context for writing like ,this is often Student $t$ tests for comparing means and (Pearson) correlations. There is some historical context for this, for example in treatments that focused on a reference case of a bivariate normal distribution with a correlation as one parameter.
So then writers start talking about regression.
These texts are usually innocent of any formal specification of a data generation process.
A: 
"Also, how did such myths come about?"

One common assumption in regression is homoscedasticity (and a myth is that this is also necessary). Transformations are used to bring the data closer to this assumption.
The violation of the assumption doesn't make the fitting method bad, least squares regression is the best unbiased linear estimator (in terms of lowest variance of the estimates) no matter what the underlying distribution is.
But, the violation of the assumptions may cause wrong inferences when we express the observed effects in terms of significance/p-values.
There is a difference between the assumptions that are necessary for least squares regression to work, and the assumptions that are necessary for the significance and hypothesis tests based on least squares regression to work.
A: There are three myths that bother me.

*

*Predictor variables need to be normal.


*The pooled/marginal distribution of $Y$ has to be normal.


*Predictor variables should not be correlated, and if they are, some should be removed.
I believe that the first two come from misunderstanding the standard assumption about normality in an OLS linear regression, which assumes that the error terms, which are estimated by the residuals, are normal. It seems that people have misinterpreted this to mean that the pooled/marginal distribution of all $Y$ values has to be normal.
Indeed, as is mentioned in a comment, we still get a lot of what we like about OLS linear regression without having normal error terms, though we do not need a normal marginal distribution of $Y$ or normal features in order to have the OLS estimator coincide with maximum likelihood estimation.
For the myth about correlated predictors, I have two hypotheses.

*

*People misinterpret the Gauss-Markov assumption about error term independence to mean that the predictors are independent.


*People think they can eliminate features to get strong performance with fewer variables, decreasing the overfitting.
I understand the idea of dropping predictors in order to have less overfitting risk without sacrificing much of the information in your feature space, but that seems not to work out. My post here gets into why and links to further reading.
A: Myth
A linear regression model can only model linear relationships between the outcome $y$ and the explanatory variables.
Fact
Despite the name, linear regression models can easily accomodate nonlinear relationships using polynomials, fractional polynomials, splines and other methods. The term "linear" in linear regression pertains to the fact that the model is linear in the parameters $\beta_0, \beta_1, \ldots$. For an in-depth explanation about the term "linear" with regards to models, I highly recommend this post.
A: @Dave's answers are excellent.  Here are some more myths.

*

*The original scale/transformation for Y is one you should use in the model.

*The central limit theorem means you don't have to worry about any of this if N is moderately large.

*Trying different transformations for Y does not distort standard errors, p-values, or confidence interval widths.

A: Myth: The error/deviation of the observations needs to be normally distributed.
No, it doesn't.
It is not just about the distribution of the errors of the observations, Instead what often matters is the distribution of the error of the estimates.
These estimates are computed as a weighted sum of the observations $$\hat\beta = M \cdot y$$ with $$M = (X^TX)^{-1}X^T$$
If we want to estimate the error or significance of the estimates $\hat\beta$, then it is sufficient if those estimates follow approximately a normal distribution. This can happen also when the sampled error of observations $y$ do not follow a normal distribution.
Due to the same principle of the central limit theorem, a statistic that is a weighted sum of variables or some sort of mean of variables will approach a normal distribution.
So even if the distribution of the error/deviation of the observations is not normally distributed, the error/deviation estimates might still be approximately normally distributed.
A: Myth: Variables that are not "significant" should be removed from a multiple regression.
See When should one include a variable in a regression despite it not being statistically significant? for a discussion.  Then search our site for "model identification," "regularization," "Lasso," etc.
A: Myth: You should always standardize (or somehow "normalize") variables for the purpose of fitting regression models.
Usually not: software will either do this automatically (under the hood, as it were) or uses algorithms that accommodate huge ranges of values among the variables without losing numerical precision.
When the order of magnitude of one explanatory variable is more than about eight times greater than of another variable, though, then watch out: even preliminary standardization can run into trouble.  ("Eight" orders of magnitude is the square root of double precision, which is about 15.6 orders of magnitude.) The commonest example is when a date is used along with other variables, because some dates are represented as the number of seconds elapsed since approximately 1970, which is on the order of $10^9$ seconds.
A: Myths:

*

*The normality of residuals (and possibly other assumptions of the model) should be tested with a formal hypothesis test, such as the Shapiro-Wilk test.

*A small $p$-value of such tests indicates that the model is invalid.

Facts:

*

*Formal test of normality (and of other assumptions such as homoskedasticity) do not answer the relevant questions and if they are used to guide subsequent actions, can distort the operating characteristic of the models (e.g. inflating type 1 errors, change distribution of $p$-values under the null etc.).

*A "significant" Shapiro-Wilk test of the residuals just indicates some degree of incompatibiltiy with a normal distribution. It does not say that the (inevitable) deviation from a normal distribution is meaningful or impactful concerning the operating characteristics of the model. Some aspects - e.g. prediction intervals - are more sensitive with regards to the distribution of the errors than others. The $t$-test of the coefficients are reasonably robust (with regards to type 1 errors), for example. Whether or not the deviation of the residuals from a normal distribution is worrysome depends on a number of things: the goal of the analysis, the sample size, the degree of deviation, and more.

A: Myth: If the histogram of the residuals is nicely bell-shaped, and if the normal q-q plot of the residuals is very close to a straight line (and the sample size is reasonably large so that sampling error is minor), then the normality assumption is reasonable.
