Suppose we have two random variables $X$ and $Y$ with unknown distributions. I am looking for an unbiased estimator for the absolute expected difference: $$ | E \{ X - Y \} | . $$ For instance, suppose we have unbiased independent observations $x_1, \ldots, x_n$ and $y_1, \ldots, y_m$ ($n,m \geq 1$), how can we use these data to construct an unbiased estimator for the above quantity?

Preferably, I would like to find an unbiased estimator for the general case, including perhaps only mild assumptions such as the existence of the first (few) moment(s). However, I am happy with any progress including:

  • solutions for specific distributions with unknown parameters (e.g., under the assumption that $X$ and $Y$ are both distributed according to a normal distribution, but the means and variances are unknown),
  • efficient biased estimators, for instance that minimize the mean squared error (again, suggestions for the general case and for specific distributions are welcome), and
  • any thoughts on good estimators for special cases.

Thanks for any input!

  • $\begingroup$ Are you looking for it to never be biased in all cases, on average, or in a specific case? $\endgroup$
    – John
    Aug 31 '11 at 7:57
  • 2
    $\begingroup$ Is there any reason you suspect the absolute value of the obvious estimator for $E[X-Y]$ would be biased? $\endgroup$
    – Nick Sabbe
    Aug 31 '11 at 9:12
  • $\begingroup$ If you plug in sample means for $X$ and $Y$ you will get a consistent estimator. Take for simplicity $m=n=1$: $E|x_1 - y_1| \geq |Ex_1 - Ey_1| = |\mu_x - \mu_y|$ the bias indeed is present, but I think it is crucial only in very small samples... $\endgroup$ Aug 31 '11 at 9:32
  • 1
    $\begingroup$ Why do you want an unbiased estimator? $\endgroup$
    – Karl
    Sep 1 '11 at 1:29
  • 3
    $\begingroup$ There seem to be some discrepancies between the question asked and the comments. Is it an estimator for $\vert E[X-Y]\vert$ that is desired (as in the question) or an estimator for $E\vert X - Y\vert ]$? It is not obvious to me that $\vert E[X-Y]\vert = \vert E[X] - E[Y]\vert = \vert \mu_X - \mu_Y \vert$ is the same as $E\vert X - Y\vert ]$. $\endgroup$ Oct 12 '11 at 1:18

Bootstrap bias correction was invented to adjusted for bias in the estimation of $f(Z)$ (my $Z$ is your $X-Y$). The idea is very simple: create $B$ bootstrap resamples from your data, and calculate $f_b=f(Z_b)$ for each one of them. Then the bootrstrap estimate of the bias is $\bar{f_b}-f(Z)$, where $\bar{f_b}$ is the average of $f_b$'s. Finally, subtract this bias from $f(Z)$ do get the bias corrected estimate $2f(Z) - \bar{f}_b$.

This estimate is unbiased, but has much more variance then the uncorrected estimate.

The reference for bootstrap methods including this bias correction is: Efron, Tibshirani 'An introduction to the bootstrap' (1993), Chapman & Hall.

  • $\begingroup$ Thanks for your answer! I had heard about bootstrapping, but have not yet read the paper by Efron and Tibshirani. However, contrary to your claim, it seems this estimate is not unbiased. Where did you get that claim from? Do you perhaps mean that it is asymptotically consistent? $\endgroup$
    – Peter
    Oct 5 '11 at 12:33
  • $\begingroup$ I'll have to check on the unbiasedness part. $\endgroup$
    – Aniko
    Oct 5 '11 at 14:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.