In Feature Engineering and Selection: A Practical Approach for Predictive Models the authors explain that on predictive modeling it is important to understand the response (or dependent) variable distribution before starting the modeling. This makes perfect sense, but what is not very clear to me is what they add later: "Understanding the distribution of the response as well as its variation provides a lower bound of the expectations of model performance. That is, if a model contains meaningful predictors, then the residuals from a model that contains these predictors should have less variation than the variation of the response". Honestly, I do not understand why this should be true (maybe they mean this result should be the goal of the modeling process?). Can anyone please provide some intuition/mathematical sketches?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.