I am confused about a certain model building technique that seems to exist, at least in practice (I am not sure whether it has its place in textbooks). Question 1: I wonder under what conditions or for what goals the following model building technique would be appropriate:

  1. Start from some sensible model (e.g. based on subject-matter knowledge or some theory)
  2. Inspect model residuals
  3. If the residuals satisfy the model assumptions (e.g. they are non-autocorrelated, homoskedastic and having the assumed distribution), consider the model appropriate
  4. If the residuals fail to satisfy the model assumptions, modify the current model attempting to remove the undesired property of the residuals (e.g. given autocorrelated residuals, explicitly allow for ARMA patterns in model errors; given ARCH patterns in residuals, allow model errors to follow a GARCH process; given a mismatch between the assumed theoretical error distribution and the realized empirical distribution of the residuals, select another theoretical error distribution) and go to step 2.

I don't think such a technique will be efficient for predictive modelling as it might be prone to overfitting. But what about descriptive or explanatory modelling? Question 2: Does this technique make sense if we want to

  • obtain an accurate description of the data generating process?
  • do hypothesis testing, e.g. assess whether a certain regressor's effect on the regressand is statistically significant?

Question 3: Can some (small) changes to this model building technique make it relevant for any of the tasks?

I did not include an assessment of individual or joint significance of regressors in the model building technique above, but in principle it could be considered, too.

  • $\begingroup$ Reply to the suggestion to include time-series tag: even though two of three examples (autocorrelated errors and ARCH-type errors) were from time series, the question is more general than that (e.g. the third example of mismatch in theoretical and empirical error/residual distribution). $\endgroup$ – Richard Hardy Nov 2 '15 at 18:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.