Here are important assumptions that one has to check when performing a linear

  • Assumption 1: Homoscedasticity of residuals or equal variance (with Breusch-Pagan test for example)
  • Assumption 2: Normalcy of residuals (with Shapiro-Wilk test for example)
  • Assumption 3: No autocorrelation of residuals (for time series with Durbin Watson test for example)

Here is my question: When performing a multiple linear regression (with more than ond independent variable), should one do each of the above-mentioned test on the entire model or should one run each of the above-mentioned test for each individual independent variable with the dependent variable?

  • $\begingroup$ VIF is only available for the entire matrix of predictors (independent variables) $\endgroup$ – ERT Jul 24 '18 at 20:24
  • $\begingroup$ Thank you @ERT you are right: I removed that point of the list $\endgroup$ – ecjb Jul 24 '18 at 20:26
  • $\begingroup$ I'd suggest to look through the many, many questions on our site on OLS assumptions. $\endgroup$ – Michael M Jul 24 '18 at 20:57
  1. You can only check the homoscedasticity of residuals after fitting your model.
  2. Same answer as 1.
  3. You can only get residuals after fitting the entire model. Thus, you need to fit your whole model before analyzing residuals.

Residuals are 'what's left' that isn't explained by your model; they're in a way an estimate of the random noise in your assumed model, i.e. the $ \epsilon $ in $$ Y = X\beta + \epsilon $$

Since we assumed all this nice stuff at the beginning in order to get nice formulas and results for linear regression, we have to double-check at the end whether or not our assumptions were reasonable.

  • $\begingroup$ Thank you for your answer @Kevin Li. However, and maybe my question we not clear enough, I got the concept of the residuals in a 2-D system (with 1 dependent and 1 independent variable). However in system with more than one independent variable (n-dimensional system where n = 3 or more), the residuals also lie in multidimensional system with respect to the fit. My question was : should those 3 above mentioned test be performed for each individual independent variable with the (only) dependent variable SEPARATELY. Or should those 3 tests be performed with all dependent variable TOGETHER? $\endgroup$ – ecjb Jul 24 '18 at 20:40
  • 1
    $\begingroup$ They should be performed with whatever combination of variables you're using for the model that you're interested in analyzing. $\endgroup$ – Kevin Li Jul 24 '18 at 21:04
  • $\begingroup$ Thank you again @Kevin Li. So you mean those 3 tests be performed with all dependent variables TOGETHER. That for example one Breusch-Pagan test for the multiple linear regression (and not one Breusch-Pagan test for each independent variable)? If yes, what is the intuition/theory behind this answer? $\endgroup$ – ecjb Jul 24 '18 at 21:10
  • 1
    $\begingroup$ You keep saying multiple independent and dependent variables, but it's uncertain to me which you actually mean. What's the dimension of your Y and what's the dimension of your X? $\endgroup$ – Kevin Li Jul 24 '18 at 21:14
  • $\begingroup$ independent variables: the predictors (on x-axis, X in your example) dependent variables: the predicted values (on y-axis, Y in your example) en.wikipedia.org/wiki/Linear_regression en.wikipedia.org/wiki/Dependent_and_independent_variables $\endgroup$ – ecjb Jul 24 '18 at 21:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.