Practical usage of the bias variance tradeoff

Question

I understand the bias-variance tradeoff. But, I have never come across a scenario where that has changed anything in the modelling process. Is there any practical scenario that you have encountered where you dug into the bias and the variance tradeoff and the way you proceeded with the modeling changed because of that?

I have been through scenarios where I have had models that had pretty different performances on different validation sets (in cross-validation) and then I reduced the model complexity as a consequence of that but that is just dealing with the observation that "maybe the model is too complex". But, I have not had any scenario where the tradeoff between bias or variance or that knowledge came in handy and led to an actual action.

Answer 1

"Although the bias-variance decomposition may provide some interesting insights into the model complexity issue from a frequentist perspective, it is of limited practical value, because the bias-variance decomposition is based on averages with respect to ensembles of data sets, whereas in practice we have only the single observed data set. If we had a large number of independent training sets of a given size, we would be better off combining them into a single large training set, which of course would reduce the level of over-fitting for a given model complexity." - Pattern Recognition and Machine Learning (Bishop).