5
$\begingroup$

I am confused about some assumptions of linear regression: homoscedasticity and residuals are normally distributed. These two require residuals, but to get the residuals, we need to fit the model first. However, before modeling, I think we should check whether the model assumptions are satisfied or not. So my questions are

  1. How to check for these assumptions before modeling?
  2. For multiple linear regression (e.g. $y = b_0 + b_1x_1 + b_2x_2$), we are testing for homoscedasticity and normality of residuals for all $x$s at once or test each $x$ one by one?
  3. If we have dataset $D$, and we split it into training set and test set, are we testing both assumptions on $D$ or on the training set only?
$\endgroup$
2
  • $\begingroup$ Please edit the question to limit it to a specific problem with enough detail to identify an adequate answer. $\endgroup$
    – Community Bot
    Commented Sep 11 at 8:23
  • 2
    $\begingroup$ You might find the distinction between the true structural model (which involves errors) and the fitted model (which involves residuals) useful: stats.stackexchange.com/questions/639705/… $\endgroup$
    – Kuku
    Commented Sep 11 at 11:03

1 Answer 1

12
$\begingroup$

The questions themselves are interesting and nontrivial enough that I believe you may have some basic knowledge about assumption testing already, so I'm not telling you what to do in particular (for which I'd refer to the literature and many places on the internet anyway).

A major thing to understand about model assumptions is that formal statistical model assumptions are never literally fulfilled in reality, and the task is not to find out whether they hold or not, but rather to find out whether there are issues with the data and assumptions that have the potential to mislead conclusions. This is a more subtle problem and in many cases hard to assess. Therefore as a standard it is usually taught to look at plots and see whether data look compatible with the model assumptions, which is arguably mostly better than not doing anything, but doesn't address exactly the right problem. I have written generally about model assumptions here and here.

Now your questions:

  1. In fact you are right that model assumptions on the residuals can only be checked by looking at the residuals, so the standard practice is to fit the model first, and then to see from the results and particularly residual plots whether there are problems. (You can see certain things from looking at the data before model fitting, but they are usually not conclusive, at least not in a situation with multiple predictors.) I assume that you understand that the model can be fitted regardless of whether model assumptions hold or not.

  2. Normality of the residuals regards the model that you are fitting, with all predictors, so it makes little sense to test this "one by one"; this is about the residuals from the model with all predictors that you are using. The situation with homoscedasticity is more subtle, because we can only detect heteroscedasticity if it plays out in such a way that we can see it as a clear "tendency", such as growing variance of the residuals along an $x$-variable. So it can make sense to check for heteroscedasticity of the residuals (from the full model) against every single predictor, but also against the fitted values (which are the "best" linear combination of the predictors), and maybe even against time (if a time order of observations is known but not part of the model); the latter can also be used to check for potential dependence over time (note that in principle you can also have dependence between residuals belonging to similar values of any $x$, so also this can be addressed looking at residuals vs. every single $x$.).

  3. This depends on what your aim is. For example it may be that your aim is the assessment of prediction quality. Note that for assessing the predictive performance on the test set of a model fitted on the training set, it is not necessary that the standard model assumptions are fulfilled (neither on test nor on training set). In particular, you don't need normality, and you don't need linearity (of course prediction quality may not be good if in particular you don't have linearity, but the test set will tell you this, so for the assessment of the fit on the test set you don't need this assumption). What you need is that the test set is representative of the predictions for which you will later use the model, which is questionable in case of dependence in particular, but also in case of potential "regime change". Note that it is not a good idea to test model assumptions on the full data set in such a case (at least assuming that you will use the model only if assumptions look fine), because this leads to "information leakage", i.e., the test set is no longer independent from the model fit on the training set as information from the test set (on model assumptions) has been used in model selection.

There is however a different use of a split of the data set. Note that there is a fundamental problem with model assumption checking, which is that the standard theory assumes that data to be analysed have not been selected based on features of the data themselves. If you only fit the model conditionally on model assumption checks, you will automatically violate this assumption, which technically means that model assumption checking itself violates the model assumptions. I call this "misspecification paradox". Arguably in many situations this is less harmful than running a model that is grossly wrong, so it doesn't mean that you should never do assumption checking, however it is an issue. One way to address this is to split the data set, to test model assumptions on one part, and to fit your regression on the other part if assumptions on the first part seem fine. A problem with this is obviously that if on top of this you want a split into test and training, you need a partition of your data set into at least three parts. So this is only recommendable if you have enough data.

$\endgroup$
1

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.