4
$\begingroup$

As far as I know, one can differentiate between two main goals of the regression analysis:

  1. The goal is understanding causal relations between variables. Here, one has to check several common regression assumptions (main being linearity, normality, residuals zero mean, homoscedasticity, independence of errors). Collinearity needs to be analysed with care and removed if possible (either by deleting collinear predictors detected by variance inflation factors or applying principal component regression etc.).

  2. The goal of the analysis is creation of predictive models. In contrast to the previous case, here one does not need to worry about collinearity at all (there are many online sources regarding this).

However, I am not sure what should one do with other "standard" regression tests, such as testing assumptions and hypotheses (p-values). Can one ignore these as well, just like collinearity, when the goal is building predictive models?

In the other words: Could some "dummy" forward or backward feature selection be applied to select the features that maximize prediction R-square (using cross-validation or test sets), without taking care of regression assumptions or predictor significance?

Improve this question
$\endgroup$
1

2 Answers 2

Reset to default
2
$\begingroup$

I think for prediction, the only thing that matters is validating properly to avoid overfitting.

By "prediction" I mean that the output of the model is a point estimate of some future response. On the other hand, if the output includes not only the point estimate but also CI, then it's different. Imagine you know the "true" predictors and true regression coefficients for a linear model, but you don't know the distribution of error terms and assume it's normal. Then your point estimate of future response will be fine, but you'll never be able to provide an adequate CI for it if the normality assumption doesn't hold.

Improve this answer
$\endgroup$
1
  • 1
    $\begingroup$ Ignoring the linearity assumption is an excellent way to create biased estimates. $\endgroup$
    – Alexis
    Commented Sep 12, 2014 at 18:19
3
$\begingroup$

@James makes a great point on estimating prediction intervals, but I don't think you want to say that the only thing that matters for point prediction is validating to avoid overfitting. Some of those regression diagnostics have clear implications for improving a model's predictive performance. One obvious case is if your model assumes a linear relationship with a predictor when the relationship is actually curvilinear then you can improve predictive performance by, for example, adding polynomial terms.

Improve this answer
$\endgroup$
2
  • $\begingroup$ The model doesn't have to be linear, but in general I agree that diagnostics could be used to come up with new predictors. $\endgroup$
    – James
    Commented Sep 12, 2014 at 18:27
  • $\begingroup$ Point prediction is actually my goal. Thanks for your inputs. Based on them, I think it is safe to proceed with nested cross validation that will do the feature selection in the inner loop (for model selection) and model assessment in the outer loop (for performance evaluation), concluding with model evaluation on holdout test set (to make sure I detect overtiffing). By repeating it for different model specifications, I can see which one works the best for given dataset (multiple linear regression with linear, interaction, quadratic terms etc.). $\endgroup$
    – Duran
    Commented Sep 17, 2014 at 7:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.