6
$\begingroup$

What can be said about the variance of the following quantity

$$ \frac{1}{n}\sum_{i=1}^n \left(\frac{f(x_i)}{\sum_{j=1}^nf(x_j)}-b\right)g(x_i) ? $$

Here, $b \in [0, 1]$, and the $x_i$s are i.i.d and $f(x) \in [0, +\infty]\; \forall x$.

In particular, for what $b$ is the variance minimized ?

Improve this question
$\endgroup$
3
  • 1
    $\begingroup$ Looks very difficult. Perhaps as a first approximation for large $n$, you could say $\sum_{j=1}^nf(X_i) \approx \mathrm{E}[f(X)]=\mathrm{E}[f]$? Then I think you could get the variance, provided you can calculate $\mathrm{E}[f]$, $\mathrm{E}[g]$, $\mathrm{E}[fg]$, $\mathrm{E}[g^2]$, $\mathrm{E}[fg^2]$ and $\mathrm{E}[f^2g^2]$. But again, it would be probably give an underestimation of the true variance. $\endgroup$
    – StijnDeVuyst
    Commented Apr 23, 2018 at 12:46
  • 2
    $\begingroup$ This question is formulated so broadly that useful solutions seem unlikely. The answer could lie anywhere in $[0,\infty)$ depending on the details of $f$, $g$, and the common distribution of the $x_i.$ $\endgroup$
    – whuber ♦
    Commented Apr 23, 2018 at 14:15
  • $\begingroup$ Thanks for the comments. @StijnDeVuyst, indeed the optimal solution features those terms you anticipated. See solution below. $\endgroup$
    – dohmatob
    Commented Apr 24, 2018 at 17:04

1 Answer 1

Reset to default
1
$\begingroup$

It turns out that for large $n$, the above estimator can be written in the form $\mathbb E_X [(h(X) - b) g(X)]$ where $h(x) := f(x) / \int f(y) d\mu_X(y)$ , and by direct computation, has variance $\mathbb E_X [h(X)g(X)] - b\mathbb E_X[g(X)^2]$ which is of course minimized by taking

$$b = b^* := \frac{\operatorname{cov}[h(X),g(X)]}{\operatorname{var}[g(X)]}. $$

This is a variance-reduction technique known as "control variates".

It remains to estimate $\operatorname{cov}[h(X),g(X)]$ and $\operatorname{var} [g(X)]$ from finite samples $x_1,\ldots,x_n$, and we're done.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.