There are usually four assumptions associated with a linear regression model: (1) linear relationship, (2) normal residuals, (3) homoscedastic residuals, and (4) i.i.d residuals. I think that it is usually said that these assumptions should be checked for the fitted model.

If we have many independent variables, would it be possible to confirm these assumptions by looking at all possible simple linear regression models and their residuals (assuming there is no multi-collinearity) instead of looking at the fitted multiple regression model?

  • $\begingroup$ What does "many" predictors mean (you say "independent variables")? The number of possible models for $p$ predictors explodes as $2^p$, which makes any such idea quickly impracticable, to say nothing of whether transformations or interaction terms are considered, or of the inferential implications of fishing in model space. $\endgroup$ – Nick Cox Sep 29 '20 at 12:37
  • $\begingroup$ @NickCox Sorry for the ambiguity, I meant that if you have $p$ predictors and one response, then you look at $p$ simple linear regression models (one for each predictor). $\endgroup$ – Vierni Sep 29 '20 at 13:18
  • $\begingroup$ It is important to distinguish "errors" (the random variables in the model) from "residuals" (the deviations between actual and fitted values). The residuals can never be normal or iid and rarely are truly homoscedastic. It is unnecessary to assume normality of the errors, too. Finally, since each possible linear regression model is a different model, with different meanings and interpretations of the coefficients, why would examining all of them be of any use in assessing the model you are really interested in? $\endgroup$ – whuber Sep 29 '20 at 13:28
  • 3
    $\begingroup$ That's a highly problematic approach. $\endgroup$ – Dave Sep 29 '20 at 13:52
  • 1
    $\begingroup$ The results from single-predictor models are of very limited relevance to those from a better multiple-predictor model. Indeed, a better multiple-predictor model must capture some of the variability missed by any single predictor model. $\endgroup$ – Nick Cox Sep 29 '20 at 14:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.