Estimator that is optimal under all sensible loss (evaluation) functions

Question

Consider a probability distribution $D$ with a parameter $\theta$ and an i.i.d. sample $S$ from that distribution. I am interested in an estimator $\hat\theta(S)$ of $\theta$ that satisfies the following condition: $$ \hat\theta(S) = \arg \min_{\hat\theta(S) \in \Theta(S)} \mathbb{E}\left( L(\hat\theta-\theta) \right) $$ for all loss functions $L$ such that $L$ is monotonically nonincreasing in $(-\infty,0)$, has a value of zero at 0, is monotonically nondecreasing in $(0,+\infty)$, and has a positive value somewhere away from zero*, where $\Theta(S)$ is a set of all possible estimators based on the sample $S$ and the expectation is taken over the possible i.i.d. samples.

Does there exist a special term for such an estimator in the statistics literature? Where could I learn more about such an estimator (conditions for its existence, its properties, and some examples)?

For instance, originally I guessed that $\hat\theta$ defined as the empirical mean of $S$ would be such an estimator when $\theta$ is the expectation and $D$ is normal. In other words, I guessed that the maximum likelihood estimator of the location parameter of the Gaussian distribution would satisfy the condition. But it seems I am wrong because for this estimator the condition cannot hold uniformly over all possible values of $\Theta(S)$, as pointed out by @CagdasOzgenc (as of now @CowboyTrader).

My question is motivated, among other, by statements such as this:

For example, in cases with a linear forecasting model and data that are jointly normal in the outcome and predictor variables, under MSE loss the forecasting model minimizes the equivalent of the negative of the maximum likelihood estimator (MLE). Assuming that the model is known and the variables are joint normally distributed, the MLE is an efficient estimator of the model parameters, regardless of the loss function. One can then proceed by simply plugging the maximum likelihood parameter estimates into the optimal forecast for the problem that still involves the correct loss function.

(Elliott and Timmermann, 2016) (emphasis is mine).

*_{$L$ could be strictly decreasing to the left of 0 and strictly increasing to the right of 0 if that makes it easier, but I would prefer a more general $L$ as above.}

References:

• Elliott, G., & Timmermann, A. (2016). Forecasting in economics and finance. Annual Review of Economics, 8, 81-110.

Answer 1

Universally Uniformly Best Unbiased Estimator

If you consider unbiased estimators and convex loss functions then you can consider the universally uniformly best unbiased estimator (UUBUE).

From "Pinelis, Iosif. A characterization of best unbiased estimators. arXiv preprint arXiv:1508.07636 (2015)."

A statistic $T$ is called universally uniformly best unbiased estimator (UUBUE) if it is $\mathcal{L}$-UBUE for all convex loss functions $\mathcal{L}$.

...

Proposition 9. Take any statistic $T$ and any loss function $\mathcal{L} \in \mathscr{C}$ . Then $T$ is a UMVUE iff $T$ is an L-UBUE iff $T$ is UUBUE.

The proof of this proposition is ascribed to L.B. Klebanov (Unbiased estimates and convex loss functions translated in 1978) and L. Schmetterer and H. Strasser (Zur Theorie der erwartungstreuen Schätzungen 1974). I can not find an online source for the latter but earlier work from Schmetterer already deals with generalizing for different loss functions than quadratic (I haven't read it to see if something similar as the proposition occurs in it)

---

Uniformly Minimum Risk Unbiased Estimator

Another term that has been used is Uniformly Minimum Risk Unbiased Estimator (UMRUE)

see:

Qiguang, Wu. "Existence of the uniformly minimum risk unbiased estimator in seemingly unrelated regression system." Acta Mathematica Sinica 11.1 (1995): 23-28.