Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It only takes a minute to sign up.
Sign up to join this community
Disguised as other questions, there are frequent questions where the OP checks for violations of model assumptions (e.g. normality, homogeneity of variance in linear regression) in models they intend to use purely for predictive purposes, not for inference.
Besides motivating the initial choice of a model, are there any reasons certain model assumptions matter for prediction?
No, many models make certain assumptions in order to be powerful predictors (e.g. what you stated for Linear Regression, feature independence in Naive Bayes). If those assumptions are violated, the estimator will be lead to make wrong predictions. This is because the estimator doesn't have the capacity to successfully model the problem at hand. In this case, you should select another, more complex estimator that has a sufficient capacity.
So why don't we always use a high-capacity estimator? Because they can easily overfit! And in this case we would forced to regularize the model, effectively lowering its capacity.
That being said, these assumptions are relevant only for model selection.
In short, if the assumptions are met it is better to select a simpler estimator, as more complex estimators might overfit. Else-wise the simple estimator won't be able to model the problem.
