Why efficiency matters?

Question

Suppose we are trying to estimate the quantity $\theta$ and we have that the estimator $\hat\theta_n$. Suppose it is efficient, i.e. is variance is the smallest among certain class of other possible estimators of $\theta$, say that this class is a class of unbiased estimators.

Efficient estimators are naturally desired, since they are the "best" in some sense. But what do we lose when we use estimator which is not efficient? Suppose we have two estimators which are asymptotically normal, then we can say that the confidence interval of the efficient estimator is narrower than the one of non-efficient one. But surely there is a better explanation than such hand-waving? Is there some quantification of what is lost?

My question was motivated by this quote by Ch. Sims found here:

Frequentist inference could be approached in the same way: Define your model, derive fully efficient estimators, pay no attention to anything else

Note that I would not like to ignite another frequentist vs Bayesian war, I just feel that there might be some deep result I am not familiar with.

Answer 1

By definition, an inefficient estimator will have larger risk for quadratic losses. Under some additional assumptions and simplifications, I suppose one might equate larger risk with inadmissibility, which would imply there is no need ever to use inefficient estimators, because there are uniformly superior estimators available. Better accounts of frequentist inference eschew such assumptions and focus on admissibility, leaving it to the user to choose an appropriate loss function and compare risk functions among the available admissible estimators. This unifies frequentist and Bayesian theory (there's nothing in the frequentist theory that precludes adopting a prior probability and in fact many admissible procedures have been discovered by doing precisely that) and allows for minimax estimation as well.

As a practical matter, as you surely know, one can equate loss of efficiency with cost of data collection: to achieve a given power in a study where the variance of the estimator scales like $1/n$, a reduction in efficiency by a factor $t \lt 1$ typically requires collecting $1/t$ times as much data. For example, the statistician who can find an estimator that is twice as efficient as one proposed by a client has (caeteris paribus) just halved the cost of the client's data collection.

There are subtleties involving asymptotic efficient estimators. For example, an AEE can be inefficient for all finite values of $n$. But I hope your question isn't bearing on this issue.