Are inconsistent estimators ever preferable? A twist

Question

The thread "Are inconsistent estimators ever preferable?" and @whuber's answer in it shows that there exists an inconsistent estimator that can outperform a reasonable consistent one for all finite $n$, for a suitable loss function. @whuber's idea for constructing an example of interest is based on finding a loss function that is minimized not at the true parameter value but elsewhere. (Edit: the last sentence is incorrect.)

I would like to take the problem one step further. In examples similar to @whuber's, there may exist a pseudo-true (for lack of a better term) parameter value that minimizes the expected value of the loss function. (This value depends on the loss function and the actual parameter value, I think.)

Question 1: If we have a reasonable consistent estimator for the pseudo-true value (corresponding to a given loss function and the actual parameter value), are there examples of an inconsistent estimator which outperforms it for all finite $n$ with respect to the same loss function?

Edit: I misread @whuber's example. There, the loss function is minimized not only elsewhere but also at the true parameter value, contrary to my initial understanding. This makes my question lose ground. However, I have a related question.

Question 2: Let us restrict the choice of loss functions so that they achieve minimum at the true parameter value but not anywhere else. (This rules out the type of loss functions used by @whuber.) If we have a reasonable consistent estimator, are there examples of an inconsistent estimator which outperforms it for all finite $n$?

Answer 1

In the previous question, the example by whuber was actually a cost function that was minimized when the estimate $t$ equals the true parameter value $t=\mu$, Namely it was zero for $\mu \leq t \leq \mu+1$, and thus at the minimum value for $t=\mu$.

Edit: The question has changed, but that example by whuber will still work, even when the minimum of the cost function is uniquely located at $t = \mu$. For instance consider this loss function:

$$L(t \vert \mu)= \begin{cases} 1 & \quad \text{if} \quad t < \mu \\ (t - \mu)^2& \quad\text{if}\quad \mu \leq t \leq \mu + 1 \\ 1 & \quad\text{if}\quad \mu + 1 < t \end{cases}$$

The consistent estimator, which will approach $\mu$ for $n \to \infty$ will result in an expectation value for the cost of $0.5$, and any estimator with a slight bias overestimating the mean with $d$ will approach $d^2$ as the expectation value for the cost.

---

Continuity

The trick of that example is that the cost function had a discontinuity at the 'true parameter'.

If instead, the cost/loss function at the true parameter value $L(\mu)$ is continuous then the consistent estimator will approach this value (by the continuous mapping theorem)

$$ \lim_{n \to \infty} L(t_n) = L(\mu)$$

Then if $L(\mu)$ is also the lowest possible value $$\forall x\neq \mu : L(\mu) \leq L(x)$$ then the consistent estimator can not be outperformed for all $n$.

This is a bit of a handwaving argument, I imagine that there might be some pathological case where the cost function for the consistent estimator and the non-consistent estimator both approach the minimum value but the consistent estimator does this faster. E.g. adjust the cost function for whuber's example to be two blocks with some small size $d$, e.g. 0 for $\mu-d<t<\mu+d$ and 0 for $\mu+0.5-2d<t<\mu+0.5+2d$ and 1 elsewhere.

Unique minimum

In the case that $$\forall x\neq \mu : L(\mu) < L(x)$$ I can not imagine these pathological cases to remain.

(But maybe you should accurately define 'consistency' and 'dominate'/'outperform' because I can imagine discrepancies there, for instance consistent estimators with infinite variance, which will not dominate the variance of the error when compared with a biased estimator with finite variance)

Example plots:

In the plot below you can see that non-biased estimator is half the time negative (and for negative values the cost function is equal to one) which is why the expected value is >0.5 for any finite sample size.

In the next plot you can see that the cost function is at a minimum for the estimator equal to the true value, But, if this is not a unique minimum for the cost function (in the example the cost is 0 for all values $\mu \leq t \leq \mu+1$) then the limit for the biased estimator can be also the minimum value. In addition, the biased estimator has for all finite values a lower expected value of the cost function (because it is in the middle of the place where the cost value is zero, whereas the unbiased estimator is at the edge where the cost function is higher).

---

As noted in the other question. It is not every consistent estimator that performs less good than a non-consistent estimator.

In the first example we can make a biased but consistent estimator by letting the bias reduce to zero as the sample size increases, and for this estimator the estimated cost can get as close to zero as we like as long as we can increase the sample size without limit.

A comment by Richard Hardy was made related to the second example

I also wish we had a better suited vocabulary to discuss these problems. Both of the estimators in your answer are "consistent", but for different targets

We could call a consistent estimator an estimator that gets as close to the target value as we like by increasing the sample size. (e.g. close measured by the variance of the difference with the target going to zero).

Then

• If the cost function has a minimum at some point different from the mean (or when the minimum is not uniquely at the mean) then it is a bit trivial that the consistent estimator for the cost function is not necessarily a consistent estimator for the parameter estimate. And we might be able to construct non-consistent estimators (with respect to the parameter estimate) that perform better with respect to the cost.
• If there is a discontinuity of the cost function at the true parameter value, then not every consistent estimator for the parameter needs to be a consistent estimator for the cost function.

Answer 2

I think the correct question here is not whether an inconsistent estimator can be better than one specific consistent estimator. With this question, you can create very lousy consistent estimators that can be beaten by lousy inconsistent estimators.

The correct question here seems to be if there aren't any consistent estimators which are guaranteed to be better than all inconsistent estimator given large enough $n$. Both examples here fail to show whether this is the case.

In both examples, we can still make the estimate arbitrarily close to the true mean. The trick is simply to start reducing the "correction" we add to the sample mean according to the sample size. The larger the sample size, the less we need to add to the sample mean to make sure we are overestimating.

Thus these examples just show that if you use a naive estimator that does not take into account the shape of the loss function, like the sample mean, which is treating underestimation and overestimation in equal footing, then you can do better with a simple naive inconsistent estimator that adds the same constant all the time.

But this doesn't mean that you can't adjust the degree of overestimation according to sample size, and thus obtain a consistent estimator that still takes the asymmetric loss function into account.

In sum, the problem of the sample mean as an estimator in these examples is not about consistency at all. And the gains of the inconsistent estimator are also not due to its inconsistency.