What can be said about the variance of the following quantity

$$ \frac{1}{n}\sum_{i=1}^n \left(\frac{f(x_i)}{\sum_{j=1}^nf(x_j)}-b\right)g(x_i) ? $$

Here, $b \in [0, 1]$, and the $x_i$s are i.i.d and $f(x) \in [0, +\infty]\; \forall x$.

In particular, for what $b$ is the variance minimized ?

  • 1
    $\begingroup$ Looks very difficult. Perhaps as a first approximation for large $n$, you could say $\sum_{j=1}^nf(X_i) \approx \mathrm{E}[f(X)]=\mathrm{E}[f]$? Then I think you could get the variance, provided you can calculate $\mathrm{E}[f]$, $\mathrm{E}[g]$, $\mathrm{E}[fg]$, $\mathrm{E}[g^2]$, $\mathrm{E}[fg^2]$ and $\mathrm{E}[f^2g^2]$. But again, it would be probably give an underestimation of the true variance. $\endgroup$ – StijnDeVuyst Apr 23 '18 at 12:46
  • 2
    $\begingroup$ This question is formulated so broadly that useful solutions seem unlikely. The answer could lie anywhere in $[0,\infty)$ depending on the details of $f$, $g$, and the common distribution of the $x_i.$ $\endgroup$ – whuber Apr 23 '18 at 14:15
  • $\begingroup$ Thanks for the comments. @StijnDeVuyst, indeed the optimal solution features those terms you anticipated. See solution below. $\endgroup$ – dohmatob Apr 24 '18 at 17:04

It turns out that for large $n$, the above estimator can be written in the form $\mathbb E_X [(h(X) - b) g(X)]$ where $h(x) := f(x) / \int f(y) d\mu_X(y)$ , and by direct computation, has variance $\mathbb E_X [h(X)g(X)] - b\mathbb E_X[g(X)^2]$ which is of course minimized by taking

$$b = b^* := \frac{\operatorname{cov}[h(X),g(X)]}{\operatorname{var}[g(X)]}. $$

This is a variance-reduction technique known as "control variates".

It remains to estimate $\operatorname{cov}[h(X),g(X)]$ and $\operatorname{var} [g(X)]$ from finite samples $x_1,\ldots,x_n$, and we're done.

| cite | improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.