If we add terms (e.g. interactions, polynomials etc.) to impove model fit suggested by residuals plots, is this classed as data dregding? In the post The role of validation in estimation and hypothesis testing Frank Harrell wrote:

If the adjustment variables were not pre-specified but determined by data dredging, you really need to use a bootstrap procedure to penalize the target variable's confidence interval. Otherwise it is been shown that all the variables in the model will have confidence intervals that are too narrow.

Would this statement apply in the case where we have improved a model fit by e.g. addding interaction terms suggested by residuals plots? Im not sure whether this is considered "data dredging"? Clearly we have changed the model so it is no longer "pre-specified".
 A: At the risk of agreeing with myself :-) I'll reinforce that any process that is supervised (i.e., makes use of $Y$), results in a bias towards something you want to brag about at the end.  Consider the simple case where there is one continuous predictor, you assume it's linear, and learn from the residual plot that linearity looks OK, so you continue to model the predictor with one coefficient.  This is completely equivalent to looking at a raw data scatterplot and looking for linearity (and other features such as constant variability of $Y$ across $X$).  If you model the variable with 1 d.f. but had given it the chance to have 2.5 degrees of freedom (e.g. your eye was able to detect something as complex as quadratic or cubic on the average), your residual variance will be underestimated and p-values and confidence interval coverage will suffer if you compute them correctly by simulation.
All of this is why Bayesian modeling has an advantage and more truthfully reports uncertainty: have a linear term with a wide prior and nonlinear terms with priors tilted towards zero, if you believe linearity was a bit more likely than nonlinearity in $X$. Bayes will then allow nonlinearity to the extent the data will allow you to reliably estimate the nonlinear effect, and the credible intervals are accounting for the uncertainties.
Very short answer: if you can't assume the model is simple up front, and you entertain the notion that it's not simple, you have to either (1) just allow it to be flexible and stick with that flexible model, (2) be Bayesian, or (3) be aggressive but bootstrap yourself to adjust for the aggression.
