6
$\begingroup$

The thread "Are inconsistent estimators ever preferable?" and @whuber's answer in it shows that there exists an inconsistent estimator that can outperform a reasonable consistent one for all finite $n$, for a suitable loss function. @whuber's idea for constructing an example of interest is based on finding a loss function that is minimized not at the true parameter value but elsewhere. (Edit: the last sentence is incorrect.)

I would like to take the problem one step further. In examples similar to @whuber's, there may exist a pseudo-true (for lack of a better term) parameter value that minimizes the expected value of the loss function. (This value depends on the loss function and the actual parameter value, I think.)

Question 1: If we have a reasonable consistent estimator for the pseudo-true value (corresponding to a given loss function and the actual parameter value), are there examples of an inconsistent estimator which outperforms it for all finite $n$ with respect to the same loss function?

Edit: I misread @whuber's example. There, the loss function is minimized not only elsewhere but also at the true parameter value, contrary to my initial understanding. This makes my question lose ground. However, I have a related question.

Question 2: Let us restrict the choice of loss functions so that they achieve minimum at the true parameter value but not anywhere else. (This rules out the type of loss functions used by @whuber.) If we have a reasonable consistent estimator, are there examples of an inconsistent estimator which outperforms it for all finite $n$?

$\endgroup$
4
  • $\begingroup$ I don't understand this question. Your characterization of a "pseudo-true parameter value" shows it to be a function of a sample, which cannot possibly be the target of any estimator. $\endgroup$
    – whuber
    May 4 '20 at 17:24
  • $\begingroup$ @whuber, this is not what I meant. I might have misquoted your example. The pseudo-true value is meant to be a population characteristic that minimizes the value of the objective function in population and that the consistent estimator targets. I have updated my post accordingly. $\endgroup$ May 4 '20 at 19:28
  • $\begingroup$ What do you mean by "value of the objective function in population"? I thought we were discussing a loss function--and by definition, that's a function of a sample. $\endgroup$
    – whuber
    May 4 '20 at 19:58
  • $\begingroup$ @whuber, sorry, I am too tired and making mistakes tonight; should have gone to bed and continued tomorrow. I have edited once more, but I will come back tomorrow and see if this needs further editing. (Also, currently I do not see why we could not take the entire population instead of a sample without violating the definition of the loss function. But that can wait until tomorrow.) $\endgroup$ May 4 '20 at 20:08
3
$\begingroup$

In the previous question, the example by whuber was actually a cost function that was minimized when the estimate $t$ equals the true parameter value $t=\mu$, Namely it was zero for $\mu \leq t \leq \mu+1$, and thus at the minimum value for $t=\mu$.

Edit: The question has changed, but that example by whuber will still work, even when the minimum of the cost function is uniquely located at $t = \mu$. For instance consider this loss function:

$$L(t \vert \mu)= \begin{cases} 1 & \quad \text{if} \quad t < \mu \\ (t - \mu)^2& \quad\text{if}\quad \mu \leq t \leq \mu + 1 \\ 1 & \quad\text{if}\quad \mu + 1 < t \end{cases}$$

The consistent estimator, which will approach $\mu$ for $n \to \infty$ will result in an expectation value for the cost of $0.5$, and any estimator with a slight bias overestimating the mean with $d$ will approach $d^2$ as the expectation value for the cost.


Continuity

The trick of that example is that the cost function had a discontinuity at the 'true parameter'.

If instead, the cost/loss function at the true parameter value $L(\mu)$ is continuous then the consistent estimator will approach this value (by the continuous mapping theorem)

$$ \lim_{n \to \infty} L(t_n) = L(\mu)$$

Then if $L(\mu)$ is also the lowest possible value $$\forall x\neq \mu : L(\mu) \leq L(x)$$ then the consistent estimator can not be outperformed for all $n$.

This is a bit of a handwaving argument, I imagine that there might be some pathological case where the cost function for the consistent estimator and the non-consistent estimator both approach the minimum value but the consistent estimator does this faster. E.g. adjust the cost function for whuber's example to be two blocks with some small size $d$, e.g. 0 for $\mu-d<t<\mu+d$ and 0 for $\mu+0.5-2d<t<\mu+0.5+2d$ and 1 elsewhere.

Unique minimum

In the case that $$\forall x\neq \mu : L(\mu) < L(x)$$ I can not imagine these pathological cases to remain.

(But maybe you should accurately define 'consistency' and 'dominate'/'outperform' because I can imagine discrepancies there, for instance consistent estimators with infinite variance, which will not dominate the variance of the error when compared with a biased estimator with finite variance)

Example plots:

In the plot below you can see that non-biased estimator is half the time negative (and for negative values the cost function is equal to one) which is why the expected value is >0.5 for any finite sample size.

If there is a dis-continuity

In the next plot you can see that the cost function is at a minimum for the estimator equal to the true value, But, if this is not a unique minimum for the cost function (in the example the cost is 0 for all values $\mu \leq t \leq \mu+1$) then the limit for the biased estimator can be also the minimum value. In addition, the biased estimator has for all finite values a lower expected value of the cost function (because it is in the middle of the place where the cost value is zero, whereas the unbiased estimator is at the edge where the cost function is higher).

If there is not a unique minimum


As noted in the other question. It is not every consistent estimator that performs less good than a non-consistent estimator.

In the first example we can make a biased but consistent estimator by letting the bias reduce to zero as the sample size increases, and for this estimator the estimated cost can get as close to zero as we like as long as we can increase the sample size without limit.

A comment by Richard Hardy was made related to the second example

I also wish we had a better suited vocabulary to discuss these problems. Both of the estimators in your answer are "consistent", but for different targets

We could call a consistent estimator an estimator that gets as close to the target value as we like by increasing the sample size. (e.g. close measured by the variance of the difference with the target going to zero).

Then

  • If the cost function has a minimum at some point different from the mean (or when the minimum is not uniquely at the mean) then it is a bit trivial that the consistent estimator for the cost function is not necessarily a consistent estimator for the parameter estimate. And we might be able to construct non-consistent estimators (with respect to the parameter estimate) that perform better with respect to the cost.
  • If there is a discontinuity of the cost function at the true parameter value, then not every consistent estimator for the parameter needs to be a consistent estimator for the cost function.
$\endgroup$
11
  • 1
    $\begingroup$ As far as I can see, discontinuity of $L$ is irrelevant. I think you lose your way in this analysis by oversimplifying the loss, which is a function of two variables: the statistic and the parameter. Moreover, there is a logical flaw: the limit does not imply the sample mean is admissible. $\endgroup$
    – whuber
    May 5 '20 at 11:32
  • 1
    $\begingroup$ It may be oversimplified but it is intentionally stripped away from technical details such that it is intuitive. The simple story is that if the cost function is such that it's value is minimum if the estimate is close/near to the true value (whether it is admissible or not) then the consistent estimator will be the only estimator that has the lowest possible value of the cost function as a limit and it should for sufficiently large $n$ dominate inconsistent estimators. For this limit it is important that the cost function is continuous so I believe that it is part of the problem. $\endgroup$ May 5 '20 at 12:39
  • 1
    $\begingroup$ In the current formulation of Richard Hardy's problem we can easily make a counterexample by using a discontinuous cost function. $$L = \begin{cases} 2 & if \quad t < \mu \\ 0 & if \quad t = \mu \\ 0.5 & if \quad \mu < t \leq \mu + 1 \\ 2 & if\quad \mu + 1 < t \end{cases}$$ In this example the discontinuity is the driving force that makes it work. I believe that continuity of the cost function should be one of the regularity conditions for the idea from Richard Hardy, about inconsistent estimators not being able to dominate a consistent estimator for all $n$. $\endgroup$ May 5 '20 at 12:57
  • 1
    $\begingroup$ Continuity doesn't matter. Little changes, as far as I can tell, if you smooth the cost function a tiny bit. $\endgroup$
    – whuber
    May 5 '20 at 15:23
  • 1
    $\begingroup$ But wouldn't in that case, if you smooth a bit, the consistent estimator 'get the cost as close' to $L=0$ as you want (if you increase $n$ sufficiently) and become better than the inconsistent estimator? In any case, the example given in this answer won't work anymore. $\endgroup$ May 5 '20 at 15:27
3
$\begingroup$

I think the correct question here is not whether an inconsistent estimator can be better than one specific consistent estimator. With this question, you can create very lousy consistent estimators that can be beaten by lousy inconsistent estimators.

The correct question here seems to be if there aren't any consistent estimators which are guaranteed to be better than all inconsistent estimator given large enough $n$. Both examples here fail to show whether this is the case.

In both examples, we can still make the estimate arbitrarily close to the true mean. The trick is simply to start reducing the "correction" we add to the sample mean according to the sample size. The larger the sample size, the less we need to add to the sample mean to make sure we are overestimating.

Thus these examples just show that if you use a naive estimator that does not take into account the shape of the loss function, like the sample mean, which is treating underestimation and overestimation in equal footing, then you can do better with a simple naive inconsistent estimator that adds the same constant all the time.

But this doesn't mean that you can't adjust the degree of overestimation according to sample size, and thus obtain a consistent estimator that still takes the asymmetric loss function into account.

In sum, the problem of the sample mean as an estimator in these examples is not about consistency at all. And the gains of the inconsistent estimator are also not due to its inconsistency.

$\endgroup$
6
  • $\begingroup$ An interesting point! I was thinking along similar lines myself. Could you construct a concrete example corresponding to the example of Sextus Empiricus where a consistent estimator that is getting progressively less biased beats the inconsistent estimator of Sextus? $\endgroup$ May 19 '20 at 5:17
  • $\begingroup$ That is a good point, but I believe that in the case with the non-uniquique minimum, the best estimator will be always be the one with the +0.5 bias. $\endgroup$ May 19 '20 at 5:39
  • $\begingroup$ In the case of the discontinuity you can make it indeed be consistent and approach zero cost, which is probably the best any estimator can do (that is, the best if you are interested in performance at infinity). $\endgroup$ May 19 '20 at 5:45
  • $\begingroup$ @RichardHardy the example of this consistent estimator is the best biased estimator. If the sample size increases then the best biased estimator will reduce in the size of bias, and eventually the bias goes to zero. $\endgroup$ May 19 '20 at 5:48
  • $\begingroup$ @SextusEmpiricus, thank you once again! Would it be possible to add such an estimator to your answer? Or I could ask a new question (though I think it makes some sense to keep the discussion within this thread as everything is interrelated). I also wish we had a better suited vocabulary to discuss these problems. Both of the estimators in your answer are "consistent", but for different targets. With respect to the target of interest, one is asymptotically biased. The notion of bias applies (or at least I would like it to apply) to finite samples. Bias can either vanish or not asymptotically. $\endgroup$ May 19 '20 at 6:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.