5
$\begingroup$

Let $X$ and $Y$ be two independent random variables. Given an (iid) random sample of size $n$ of $X$ and a random sample of size $n$ of $Y$, what is a good way to estimate the mean of their product, $E[XY]$?

The most obvious estimator is the sample mean: multiply each $X_i$ and $Y_i$, and then average over $n$.

Another unbiased estimator would be to take the average of the $X_i$'s, then take the average of the $Y_i$'s, and finally multiply these two averages.

Which of these two unbiased estimators is better (is less noisy or has lower variance)? Is there an estimator that has even smaller variance than the two proposed?

$\endgroup$
3
  • $\begingroup$ @StubbornAtom are you asking about the applications or use cases of this? $\endgroup$
    – rishai
    Jun 27, 2020 at 19:48
  • $\begingroup$ You haven't said what quantity you are trying to estimate. $XY$ is a random variable. Do you want its mean or median or variance or the whole distribution or what? $\endgroup$ Jun 27, 2020 at 23:16
  • $\begingroup$ @ThomasLumley I understand now - my mistake. The parameter of interest is the expectation/mean. I've updated the question - thanks. $\endgroup$
    – rishai
    Jun 27, 2020 at 23:36

1 Answer 1

6
$\begingroup$

I'm assuming what you want to estimate is $E[XY]$ (you don't say, but the use of the sample mean suggests it)

Intuitively, $\overline{XY}$ would work even if $X$ and $Y$ weren't independent, so it should be less efficient under the additional assumption that they are independent. Let's see how that goes

Let's look at the case where $X$ and $Y$ are Normal, to start off. The maximum likelihood estimators of the means $\mu_x$ and $\mu_y$ of $X$ and $Y$ are the sample averages $\bar X$ and $\bar Y$, and the invariance principle for MLEs says that the MLE of $\mu_x\mu_y$ is $\bar X\bar Y$.

The mean of $\bar X\bar Y$ is $\mu_x\mu_u$ (by independence). Its variance is $\mu^2_x\sigma^2_y/n+\mu^2_y\sigma^2_x/n+\sigma^2_x\sigma^2/n^2$

The mean of $\overline{XY}$ is $\mu_x\mu_y$. The variance of $XY$ is $\mu^2_x\sigma^2_y+\mu^2_y\sigma^2_x+\sigma^2_x\sigma^2$ so the variance of $\overline{XY}$ is $(\mu^2_x\sigma^2_y+\mu^2_y\sigma^2_x+\sigma^2_x\sigma^2)/n$ which is larger than the variance of $\bar X\bar Y$.

The mean and variance analysis still works when $X$ and $Y$ are not Normal, so it's still true that $\bar X\bar Y$ is more efficient. However, it's now possible that there are more efficient estimators, because the sample average is no longer the MLE. For example, if $X$ and $Y$ have a Laplace distribution, the sample medians are the MLEs of the means of $X$ and $Y$, so the product of the sample medians will be a more efficient estimator than $\bar X\bar Y$.

In the nonparametric model where all you know about $X$ and $Y$ is that they have finite means, the sample average is efficient (because basically anything else is inconsistent) and $\bar X\bar Y$ will be optimal again.

$\endgroup$
5
  • $\begingroup$ @ Thomas Lumley I'm doubly grateful for this, as it not only shows the second estimator has lower variance than the first, but the second is also one that has minimum variance under certain conditions, such as when there is no prior knowledge about the distributions. I'd like to clarify a few things in your explanation, if you don't mind. I'm only familiar with the version of the invariance principle involving a function of one argument: given an MLE for $\Theta$, find the MLE for $g(\Theta)$. $\endgroup$
    – rishai
    Jun 29, 2020 at 8:46
  • $\begingroup$ For us, the product is a function of two arguments; is there a version that treats this more general case? $\endgroup$
    – rishai
    Jun 29, 2020 at 8:46
  • $\begingroup$ Secondly, would you remember a reference for the MLE for the mean of the Laplace distribution? I haven't yet found an explanation or derivation in the texts I looked. $\endgroup$
    – rishai
    Jun 29, 2020 at 8:58
  • 1
    $\begingroup$ Think of the product as a function of one vector argument ($\bar X, \bar Y$) estimating $(\mu_x,\mu_y)$, and the function is $f(\theta)=\theta_1\times\theta_2$. $\endgroup$ Jun 29, 2020 at 21:09
  • 1
    $\begingroup$ And this question derives the Laplace MLE: math.stackexchange.com/questions/240496/… $\endgroup$ Jun 29, 2020 at 21:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.