Regression Modeling Strategies

The purpose of this category is to refer to questions and discussions about regression modeling strategies, especially when multiple methods are being combined. For example how much data reduction should be done before using $Y$? What is best practice for model validation for specific model types? How does the choice of predictive accuracy measures impact model validation? How should parameters be assigned for various parts of a model, and how does the number of parameters assigned to one part of the model affect the number of parameters to assign to another part? What is the best way to detect that parameters in a model are hard to disentangle and how could pre-modeling data reduction have helped? What is a good strategy for getting a complex model accepted by non-statisticians? When does one use traditional multivariable regression modeling vs. a black box?