Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

What are the usual assumptions for linear regression?

Do they include:

  1. a linear relationship between the independent and dependent variable
  2. independent errors
  3. normal distribution of errors
  4. homoscedasticity

Are there any others?

share|improve this question
I found this page quite helpful, for a review of the assumptions and the ways of testing them. – George Dontas Oct 3 '11 at 8:16
You can find a rather complete list in William Berry's little book on "Understanding Regression Assumptions":… – user5644 Oct 3 '11 at 9:43
While respondents have listed some good resources, it is a difficult question to answer in this format, and (many) books have been devoted solely to this topic. There is no cook book, nor should there be given the potential variety of situations that linear regression could encompass. – Andy W Oct 3 '11 at 12:49
Technically, (ordinary) linear regression is a model of the form $\mathbb{E}[Y_i] = \mathbf{X}_i \beta$, $Y_i$ iid. That simple mathematical statement encompasses all the assumptions. This leads me to think, @Andy W, that you may be interpreting the question more broadly, perhaps in the sense of the art and practice of regression. Your further thoughts about this might be useful here. – whuber Oct 3 '11 at 14:41
@mp You're right; I should have more carefully stated that $Y_i - \mathbf{X}_i\beta$ are iid. – whuber Oct 4 '11 at 13:45

The answer depends heavily on how do you define complete and usual. Suppose we write linear regression model in the following way:


where $\mathbf{x}_i$ is the vector of predictor variables, $\beta$ is the parameter of interest, $y_i$ is the response variable, and $u_i$ are the disturbance. One of the possible estimates of $\beta$ is the least squares estimate:

$$\hat\beta=\textrm{argmin}_{\beta}\sum(y_i-\mathbf{x}_i\beta)^2=\left(\sum \mathbf{x}_i\mathbf{x}_i'\right)^{-1}\sum \mathbf{x}_iy_i$$

Now practically all of the textbooks deal with the assumptions when this estimate $\hat\beta$ has desirable properties, such as unbiasedness, consistency, efficiency, some distributional properties, etc.

Each of these properties requires certain assumptions, which are not the same. So the better question would be to ask which assumptions are needed for wanted properties of the LS estimate.

The properties I mention above require some probability model for regression. And here we have the situation where different models are used in different applied fields.

The simple case is to treat $y_i$ as an independent random variables, with $\mathbf{x}_i$ being non-random. I do not like the word usual, but we can say that this is the usual case in most applied fields (as far as I know).

Here is the list of some of the desirable properties of statistical estimates:

  1. The estimate exists
  2. Unbiasedness: $E\hat\beta=\beta$
  3. Consistency: $\hat\beta\to \beta$ as $n\to\infty$ ($n$ here is the size of a data sample)
  4. Efficiency: $Var(\hat\beta)$ is smaller than $Var(\tilde\beta)$ for alternative estimates $\tilde\beta$ of $\beta$
  5. The ability to either approximate or calculate the distribution function of $\hat\beta$


Existence property might seem weird, but it is very important. In the definition of $\hat\beta$ we invert the matrix

$$\sum \mathbf{x}_i\mathbf{x}_i'$$

It is not guaranteed that the inverse of this matrix exists for all possible variants of $\mathbf{x}_i$. So we immediately get our first assumption:

Matrix $$\sum \mathbf{x}_i\mathbf{x}_i'$$ should be of full rank, i.e. invertible.


We have

$$E\hat\beta=\left(\sum \mathbf{x}_i\mathbf{x}_i\right)^{-1}\left(\sum \mathbf{x}_iEy_i\right)=\beta,$$



We may number it the second assumption, but we may have stated it outright, since this is one of the natural ways to define linear relationship.

Note that to get unbiasedness we only require that $Ey_i=\mathbf{x}_i\beta$ for all $i$, and $\mathbf{x_i}$ are constants. Independence property is not required.


For getting the assumptions for consistency we need to state more clearly what do we mean by $\to$. For sequences of random variables we have different modes of convergence: in probability, almost surely, in distribution and p-th moment sense. Suppose we want to get the convergence in probability. We can use either law of large numbers or directly the Chebyshev inequality (we employ the fact that $E\hat\beta=\beta$):

$$P(|\hat\beta-\beta|>\varepsilon)\le \frac{Var(\beta)}{\varepsilon^2}$$

Since convergence in probability means that the left hand term must vanish for any $\varepsilon>0$ as $n\to\infty$, we need that $Var(\beta)\to 0$ as $n\to\infty$. This is perfectly reasonable since with more data the precision with which we estimate $\beta$ should increase.

We have that

$$Var(\hat\beta)=\left(\sum \mathbf{x}_i\mathbf{x}_i'\right)^{-1}\sum_i\sum_j\mathbf{x}_i\mathbf{x}_j'cov(y_i,y_j)\left(\sum \mathbf{x}_i\mathbf{x}_i'\right)^{-1}$$

Independence ensures that $cov(y_i,y_j)=0$, hence the expression simplifies to

$$Var(\hat\beta)=\left(\sum \mathbf{x}_i\mathbf{x}_i'\right)^{-1}\sum_i\mathbf{x}_i\mathbf{x}_i'var(y_i)\left(\sum \mathbf{x}_i\mathbf{x}_i'\right)^{-1}$$

Now assume $Var(y_i)=const$, then

$$Var(\hat\beta)=\left(\sum \mathbf{x}_i\mathbf{x}_i'\right)^{-1}Var(y_i)$$

Now if we additionaly require that $\frac{1}{n}\sum \mathbf{x}_i\mathbf{x}_i'$ is bounded for each $n$ we immediately get

$$Var(\beta)\to 0, n\to\infty.$$

So to get the consistency we assumed that there is no autocorrelation ($cov(y_i,y_j)=0$), t the variance $Var(y_i)$ is constant and $\mathbf{x}_i$ do not grow too much. The first assumption is satisfied if $y_i$ comes from independent sample.


The classic result is the Gauss-Markov theorem. The conditions for it is exactly the first two conditions for consistency and the condition for unbiasedness.

Distributional properties

If $y_i$ are normal we immediately get that $\hat\beta$ is normal since it is a linear combination of normal random variables. If we assume previous assumptions of independence, uncorrelatedness and constant variance we get that

$$\hat\beta\sim N\left(\beta,\sigma^2\left(\sum \mathbf{x}_i\mathbf{x}_i'\right)^{-1}\right)$$

where $Var(y_i)=\sigma^2$.

If $y_i$ are not normal, but independent we can get approximate distribution of $\hat\beta$ thanks to central limit theorem. For this we need to assume that

$$\lim_n \frac{1}{n}\sum\mathbf{x}_i\mathbf{x}_i'\to A$$

for some matrix $A$. The constant variance for assymptotic normality is not required if we assume that

$$\lim_n\frac{1}{n}\sum\mathbf{x}_i\mathbf{x}_i'Var(y_i)\to B$$

Note that with constant variance of $y$ we cant that $B=\sigma^2A$. The central limit theorem then gives us the following result:

$$\sqrt{n}(\hat\beta-\beta)\to N(0,A^{-1}BA^{-1}).$$

So from this we see that independence and constant variance for $y_i$ and certain assumptions for $\mathbf{x}_i$ gives us a lot of useful properties for LS estimate $\hat\beta$.

The thing is that these assumptions can be relaxed. For example we required that $\mathbf{x}_i$ are not random variables. This assumption is not feasible in econometric applications. If we let $\mathbf{x}_i$ be random, we can get similar results if use conditional expectations and take into account the randomness of $\mathbf{x}_i$. The independence assumption also can be relaxed. We already demonstrated that sometimes only uncorrelatedness is needed. Even this can be further relaxed and it is still possible to show that the LS estimate will be consistent and assymptoticaly normal. See for example White's book for more details.

share|improve this answer
This is a really clear, excellent answer!! – Patrick S. Forscher Dec 5 '12 at 20:33
great answer..... – hypfco May 30 '15 at 19:58

There are a number of good answers here. It occurs to me that there is one assumption that has not been stated however (at least not explicitly). Specifically, a regression model assumes that $\mathbf X$ (the values of your explanatory / predictor variables) is fixed and known, and that all of the uncertainty in the situation exists within the $Y$ variable. In addition, this uncertainty is assumed to be sampling error only.

Here are two ways to think about this: If you are building an explanatory model (modeling experimental results), you know exactly what the levels of the independent variables are, because you manipulated / administered them. Moreover, you decided what those levels would be before you ever started gathering data. So you are conceptualizing all of the uncertainty in the relationship as existing within the response. On the other hand, if you are building a predictive model, it is true that the situation differs, but you still treat the predictors as though they were fixed and known, because, in the future, when you use the model to make a prediction about the likely value of $y$, you will have a vector, $\mathbf x$, and the model is designed to treat those values as though they are correct. That is, you will be conceiving of the uncertainty as being the unknown value of $y$.

These assumptions can be seen in the equation for a prototypical regression model: $$ y_i = \beta_0 + \beta_1x_i + \varepsilon_i $$ A model with uncertainty (perhaps due to measurement error) in $x$ as well might have the same data generating process, but the model that's estimated would look like this: $$ y_i = \hat\beta_0 + \hat\beta_1(x_i + \eta_i) + \hat\varepsilon_i, $$ where $\eta$ represents random measurement error. (Situations like the latter have led to work on errors in variables models; a basic result is that if there is measurement error in $x$, the naive $\hat\beta_1$ would be attenuated--closer to 0 than its true value, and that if there is measurement error in $y$, statistical tests of the $\hat\beta$'s would be underpowered, but otherwise unbiased.)

One practical consequence of the asymmetry intrinsic in the typical assumption is that regressing $y$ on $x$ is different from regressing $x$ on $y$. (See my answer here: What is the difference between doing linear regression on y with x versus x with y? for a more detailed discussion of this fact.)

share|improve this answer
What does it mean "fixed" | "random" in plain language? And how to distinguish between fixed and random effects(=factors)? I think that in my design there is 1 fixed known factor with 5 levels. Right? – stan Dec 9 '12 at 10:22
@stan, I recognize your confusion. Terminology in stats is often confusing & unhelpful. In this case, "fixed" is not quite the same as the fixed in 'fixed effects & random effects' (although they are related). Here, we're not talking about effects--we're talking about the $X$ data, ie your predictor / explanatory variables. The easiest way to understand the idea of your $X$ data being fixed is to think of a planned experiment. Before you have done anything, when you're designing the experiment, you decide what the levels of your explanatory will be, you don't discover them along the way. – gung Dec 9 '12 at 16:00
W/ predictive modeling, that's not quite true, but we will treat our $X$ data that way in the future, when we use the model to make predictions. – gung Dec 9 '12 at 16:00
Why do the βs and the ε have a hat in the bottom equation, but not in the top one? – user1205901 Apr 14 '15 at 8:41
@user1205901, the top model is of the data generating process, the bottom is your estimate of it. – gung Apr 14 '15 at 14:13

A comprehensive list of the assumption of the classical linear regressionmodel can be found here:

It is important to notice that the consequences in terms of validity of your model depend on which assumption is violated.

share|improve this answer

We're looking for long answers that provide some explanation and context. Don't just give a one-line answer; explain why your answer is right, ideally with citations. Answers that don't include explanations may be removed.

Can you add a summary of the information contained in the link? The problem with link-only answers is that they can suffer from "link rot" - we are trying to build a long-term resource, and there's no guarantee the link will work in a few years time. Otherwise links are best suited to comments rather than answers. – Silverfish Oct 4 '15 at 11:49
Nonetheless, it is a good link, while note gone. – Marcelo Ventura May 8 at 23:04

As an aside, the four assumptions the OP mentions can easily be remembered using the acronym LINE:

  • Linearity
  • Independence
  • Normality
  • Equal Variance
share|improve this answer

It's all about what you wanna do with your model. Imagine if your errors were all positively skewed/non-normal. If you wanted to make a prediction interval, you could do better than using the t-distribution.. If your variance is smaller at smaller predicted values, again, you'd be making a prediction interval that's too big..

It's better to understand why the assumptions are there.

share|improve this answer

The following are the assumptions of Linear Regression analysis.

Correct specification. The linear functional form is correctly specified.

Strict exogeneity. The errors in the regression should have conditional mean zero.

No multicollinearity. The regressors in X must all be linearly independent.

Homoscedasticity which means that the error term has the same variance in each observation.

No autocorrelation: the errors are uncorrelated between observations.

Normality. It is sometimes additionally assumed that the errors have normal distribution conditional on the regressors.

I.i.d observations: $(x_i, y_i)$ is independent from, and has the same distribution as, $(x_j, y_j)$ for all $i\neq j$.

For more information visit this page.

share|improve this answer
Rather than "no multicolinearity" I'd say "no linear dependence". Collinearity is often used as a continuous rather than categorical measure. It is only strict or exact collinearity that is forbidden. – Peter Flom Oct 3 '11 at 11:55
What about time series regression? What about generalised least squares? Your list reads a bit like list of commandments when in fact last 4 assumptions can be too restrictive if we only care about consistency and asymptotic normality of least squares estimate. – mpiktas Oct 3 '11 at 12:41
Multicollinearity raises problems of interpretation (related to identifiability of some parameters) but it definitely is not a standard assumption of linear regression models. Near multicollinearity is primarily a computational problem but also raises similar issues of interpretation. – whuber Oct 3 '11 at 14:37
@whuber & Peter Flom: As I read in the book of Gujarati at page no. 65-75. It count the "no multicollinearity" as a assumption of regression analysis. – love-stats Oct 3 '11 at 16:59
@mpiktas: If you visit the given URL in the answer then you will find assumption about time series regression. – love-stats Oct 3 '11 at 17:00

The assumption of linearity is that the model is linear in the parameters. It is fine to have a regression model with quadratic or higher order effects as long as the power function of the independent variable is part of a linear additive model. If the model does not contain higher order terms when it should, then the lack of fit will be evident in the plot of the residuals. However, standard regression models do not incorporate models in which the independent variable is raised to the power of a parameter (although there are other approaches that can be used to evaluate such models). Such models contain non-linear parameters.

share|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.