When is having an unbiased estimator important?

We have a few questions and answers about when one would prefer a biased estimate over a unbiased one, but I have not found anything on the reverse question:

In what situations is it important to consider only unbiased estimators?

A lot of emphasis is put on the concept of unbiasedness is introductory statistical courses, but I've never read a compelling defense of this. Since we generally only collect data once, when is it useful to be correct on average (besides the possible psychological comfort it provides)? In what situations would one need to be correct on average?

I'm open to philosophical arguments, but would prefer concrete examples from research or industry.

• It's difficult to imagine any definite answer could exist, for the simple reason that although an unbiased estimator is well-quantified--its bias is zero--a biased estimator is not: its bias could be anything. Where do you draw the line? Would an estimator with a bias of $10^{-100}\%$ be unacceptable compared to an unbiased estimator? Of course not. – whuber Jul 13 '17 at 18:56
• Are you taking issue with my use of the word "need"? How about "in what situations would one have a strong compelling desire to be correct on average"? – Matthew Drury Jul 13 '17 at 19:11
• Same comment: any circumstance will come with a nonzero amount of acceptable error. If anyone claims they do have a compelling need to be correct on average, you could prove them wrong by showing that they will do just as well by being $\epsilon\%$ away from average for a sufficiently small nonzero $\epsilon$. This suggests it would be better to ask questions like "What properties do unbiased estimators enjoy? How much could those properties be improved by considering biased estimators and what would be the cost of carrying out such improvements?" – whuber Jul 13 '17 at 20:01
• There's definitely compelling evidence when not to use unbiased estimators: en.wikipedia.org/wiki/… – Alex R. Jul 13 '17 at 20:13
• +1 but i still feel this question is too high level... – Haitao Du Jul 14 '17 at 2:30

I think it's safe to say there's no situation when one needs an unbiased estimator; for example, if $\mu = 1$ and we have $E[\hat \mu] = \mu + \epsilon$, there has got be an $\epsilon$ small enough that you cannot possibly care.
Of course, my comment about truly unbiased still remains: if I had an estimator had expected value of the out-of-sample error + $\epsilon$, I would happily use it instead for small enough $\epsilon$. But the method of cross-validation is motivated by trying to get an unbiased estimator of the out-of-sample error. And without cross-validation, the field of machine learning would look completely different than it does now.