I'm trying to derive the variance for a control variate estimator, but I seem to be missing a term that allows me to end up with the covariance in the final answer.

Let $f(x)$ be my function and let $h(x)$ be my control variate with $x \sim p(x)$. If I define the surrogate function:

$\tilde{f}(x) = f(x) - \beta (h(x) - \mathbb{E}_x[h(x)])$

I'd like to derive the variance of an estimator of the surrogate function:

enter image description here

As you can see, the middle term should be $Cov(f, h)$. What mistake am I making?

Also, how are there not tags for "control-variates" or "variance-reduction"?

  • $\begingroup$ With respect to your last question, this is a statistics site, not a simulation / programming site! $\endgroup$ – jbowman Jan 10 at 0:39
  • $\begingroup$ $\mathbb{E}[f(h-\mathbb{E}[h])] = \text{Cov}(f,h)$, so I'm not sure where you are seeing a mistake. $\endgroup$ – jbowman Jan 10 at 0:44
  • $\begingroup$ @jbowman , I thought $$Cov(f, h) = \mathbb{E}[(f-\mathbb{E}[f])(h-\mathbb{E}[h])]$$. What happened to subtracting the expected value of $f$? $\endgroup$ – Rylan Schaeffer Jan 10 at 1:25
  • $\begingroup$ Also, how are control variates and variance reduction not related to statistics? Topics like these are clearly related to estimators and estimator properties. $\endgroup$ – Rylan Schaeffer Jan 10 at 1:26
  • $\begingroup$ Control variates are pretty much a special-purpose simulation tool, and as such are going to, typically, be discussed within the context of programming and simulation rather than in the context of data analysis or math. Your question appears on-topic to me, but the reason there isn't a control variate tag is likely that few people with control variate questions come here, instead going to stackoverflow or someplace else. $\endgroup$ – jbowman Jan 10 at 1:47

Your derivation is correct; the two expressions for $\text{Cov}(f,h)$ referred to in the comments are the same:

$$\mathbb{E}[f(h-\mathbb{E}h)] = \mathbb{E}[fh]-\mathbb{E}[f\mathbb{E}h] \\ = \mathbb{E}[fh]-\mathbb{E}f\mathbb{E}h$$

where the last line follows as $\mathbb{E}h$ is a constant, and the expectation of an r.v. times a constant is just the constant times the expectation.

$$\mathbb{E}[(f-\mathbb{E}f)(h-\mathbb{E}h)] = \mathbb{E}[fh] -\mathbb{E}[f\mathbb{E}h]-\mathbb{E}[h\mathbb{E}f]+\mathbb{E}f\mathbb{E}h \\ =\mathbb{E}[fh] -\mathbb{E}f\mathbb{E}h-\mathbb{E}h\mathbb{E}f+\mathbb{E}f\mathbb{E}h \\ =\mathbb{E}[fh]-\mathbb{E}f\mathbb{E}h$$

Surprising, but true!

| cite | improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.