Under which conditions would a researcher choose optimally when there is a trade-off between the variance and bias of an estimator? I hope this question is not too broad...
Any help would be appreciated.
As an example, what criteria would I use, if we're referring to Linear Models?
The short answer is that it depends on your particular problem and domain. The existence of a trade-off, and there is not always one, would lead us to ask,
how much variance am I reducing
how much bias am I introducing
If you can address these questions to your satisfaction (through some cross-validations or simulations), then you can answer the question as appropriate to you.
It all depends on your objectives as a researcher. If you care a lot about getting the right answer then you may not be willing to accept any bias, as in OLS and most related models. However, there are many situations in which you might prefer to get as close as you can to the right answer, where closeness is the priority and unbiasedness is not. In this case you can often do better by minimizing the mean square error, which is the sum of the variance and the squared bias, than you could by searching for the minimum variance unbiased estimator, which is what most simple regression models do.
The question is very broad, so maybe this discussion can help narrow down what you're looking for.
