Consistency is obviously a natural and important property of estimators, but are there situations where it may be better to use an inconsistent estimator rather than a consistent one?

More specifically, are there examples of an inconsistent estimator which outperforms a reasonable consistent estimator for all finite $n$ (with respect to some suitable loss function)?