# About the variance of a weighted sum

What can be said about the variance of the following quantity

$$\frac{1}{n}\sum_{i=1}^n \left(\frac{f(x_i)}{\sum_{j=1}^nf(x_j)}-b\right)g(x_i) ?$$

Here, $b \in [0, 1]$, and the $x_i$s are i.i.d and $f(x) \in [0, +\infty]\; \forall x$.

In particular, for what $b$ is the variance minimized ?

• Looks very difficult. Perhaps as a first approximation for large $n$, you could say $\sum_{j=1}^nf(X_i) \approx \mathrm{E}[f(X)]=\mathrm{E}[f]$? Then I think you could get the variance, provided you can calculate $\mathrm{E}[f]$, $\mathrm{E}[g]$, $\mathrm{E}[fg]$, $\mathrm{E}[g^2]$, $\mathrm{E}[fg^2]$ and $\mathrm{E}[f^2g^2]$. But again, it would be probably give an underestimation of the true variance. – StijnDeVuyst Apr 23 '18 at 12:46
• This question is formulated so broadly that useful solutions seem unlikely. The answer could lie anywhere in $[0,\infty)$ depending on the details of $f$, $g$, and the common distribution of the $x_i.$ – whuber Apr 23 '18 at 14:15
• Thanks for the comments. @StijnDeVuyst, indeed the optimal solution features those terms you anticipated. See solution below. – dohmatob Apr 24 '18 at 17:04

It turns out that for large $n$, the above estimator can be written in the form $\mathbb E_X [(h(X) - b) g(X)]$ where $h(x) := f(x) / \int f(y) d\mu_X(y)$ , and by direct computation, has variance $\mathbb E_X [h(X)g(X)] - b\mathbb E_X[g(X)^2]$ which is of course minimized by taking
$$b = b^* := \frac{\operatorname{cov}[h(X),g(X)]}{\operatorname{var}[g(X)]}.$$
It remains to estimate $\operatorname{cov}[h(X),g(X)]$ and $\operatorname{var} [g(X)]$ from finite samples $x_1,\ldots,x_n$, and we're done.