When comparing two estimators, say $T_1$ and $T_2$, what does it mean by saying $T_1$ is more efficient than $T_2$?

Could someone give an easy but very concrete example?

Also I have another question about relative efficiency: https://en.wikipedia.org/wiki/Efficient_estimator

It says in the above Wikipedia article that: If $T_1$ and $T_2$ are estimators for the parameter $\theta$, then $T_1$ is said to dominate $T_2$ if:

1) its mean square is smaller for at least some value of $\theta$

2) the MSE does not exceed that of $T_2$ for any value of $\theta$

Formally $T_1$ dominates $T_2$ if

$E(T_1-\theta)^2 \leq E(T_2-\theta)^2$

I have two questions:

1) If we don't know $\theta $, then how can we show one is smaller than the other in the above inequality.

2) Also I thought there is a SINGLE "true" value of the parameter $\theta$, is it correct? then what does it mean by saying "for SOME value of $\theta$" in the above statement in Wikipedia? If there is only ONE, why does it say "for SOME" value of $\theta$?


I am just wondering, when comparing two estimator says T1 and T2, what does it mean by saying T1 is more efficient than T2


For an unbiased estimator, efficiency is the precision of the estimator (reciprocal of the variance) divided by the upper bound of the precision (which is the Fisher information). Equivalently, it's the lower bound on the variance (the Cramer-Rao bound) divided by the variance of the estimator.

The relative efficiency of two unbiased estimators is the ratio of their precisions (the bound cancelling out)

When you're dealing with biased estimators, relative efficiency is defined in terms of the ratio of MSE.

Could someone give an easy but very concrete example.

Compare the sample mean ($\bar{x}$) and sample median ($\tilde{x}$) when trying to estimate $\mu$ at the normal.

They're both unbiased so we need the variance of each. The variance of the median for odd sample sizes can be written down from the variance of the $k$th order statistic but involves the cdf of the normal. In large samples $\frac{n}{\sigma^2}\text{ Var}(\tilde{\tilde{x}})$ approaches the asymptotic value reasonably quickly, so people tend to focus on the asymptotic relative efficiency.

The asymptotic relative efficiency of median vs mean as an estimator of $\mu$ at the normal is the ratio of variance of the mean to the (asymptotic) variance of the median when the sample is drawn from a normal population.

This is $\frac{\sigma^2/n}{2\pi \sigma^2/(4 n)} = 2/\pi\approx 0.64$

There's another example discussed here: Relative efficiency: mean deviation vs standard deviation

If we don't know θ, then how can we show one is smaller than the other in the above inequality.

Generally the MSE's will be some function of $\theta$ and $n$ (though they may be independent of $\theta$). So at any given $\theta$ you can compute their relative size.

Also I thought there is a SINGLE "true" value of the parameter θ, is it correct?

Yes, at least in the usual situations we'd be doing this in and assuming a frequentist framework.

then what does it mean by saying "for SOME value of θ" in the above statement [...] if there is only ONE, why it says "for SOME" value of θ

When you are comparing estimators you want ones that do well for every value of $\theta$. If you don't know what $\theta$ is (if you did, you wouldn't have to bother with estimators), it would be good if it worked well for whatever value you have.

  • $\begingroup$ Thanks a lot for your explanation Mr Glen. But I am just wondering could you explain in layman term what exactly it means by the number 0.64 here. $\endgroup$ – john_w Jun 8 '18 at 6:18
  • $\begingroup$ Also when you said for large sample, the $\frac{n}{\sigma^2}Var(\tilde{\mu})$, does the $\tilde{\mu}$ here means the median of the sample ? $\endgroup$ – john_w Jun 8 '18 at 6:23
  • 1
    $\begingroup$ 1. You can get about as precise an estimate using a sample mean to estimate a population mean (given large random samples from a normal population) with only 64% as much data as you'd need if you estimated it using the median. 2. Oh, actually, I should have $\tilde{x}$ for the sample median, rather than $\tilde{\mu}$ (which is one way to denote the population median). Or to be even more precise, I should really have $\tilde{X}$ to denote the estimator (clarifying it is a random variable) rather than $\tilde{x}$ (a value obtained on a specific sample). $\endgroup$ – Glen_b -Reinstate Monica Jun 8 '18 at 8:38

T1 will be more efficient if it has a small variance compered to T2

  • 1
    $\begingroup$ True, but can you provide the concrete example that is requested? $\endgroup$ – whuber Dec 29 '19 at 20:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.