7
$\begingroup$

Is there a benefit of combining control variates with antithetic variates and if so how should it be done ?

In my specific case I would like to add control variates to the formulation in this paper : https://arxiv.org/abs/1804.02395, section 2.1. Briefly, the idea there is to use the antithetic variates to reduce the variance on the expected gradient of the return.

At the same time, control variates is widely used for policy gradient algorithms. How can I benefit from both approaches ?

My initial thought was to add a baseline to each of the antithetic variables.

$\endgroup$
1
  • $\begingroup$ @Xi'an Could you please explain that? The condition for variance reduction is that the antithetic variates are negatively correlated. If I subtract constants, then this should not change their correlation, right? $\endgroup$
    – ganto
    Aug 22 at 10:27
3
$\begingroup$

Using antithetic variates to improve the Monte Carlo approximation of $\mathbb E^F[h(X)]$ mean generating correlated realisations from $F$, $X_1,\ldots,X_n$ such that$$\text{var}(h(X_1)+\cdots+h(X_n))<\text{var}(h(X_1))+\cdots+\text{var}(h(X_n))\tag{1}$$ While the idea is appealing, it is difficult to implement in realistically complex settings since establishing the reduction of variance for a given $h$ [or a collection of $h$'s] is challenging.

Assuming such an antithetic scheme (1) has been constructed, if a control variate is available for the model, i.e. a function $h_0(\cdot)$ such that $\mathbb E^F[h_0(X)]=0$ and $\text{corr}(h(X),h_0(X))\ne 0$, the (overall) negative correlation between the $h(X_i)$'s does not automatically transfer to an (overall) negative correlation between the $h(X_i)+\alpha h_0(X_i)$'s. Hence, even if $\alpha$ is chosen such that $$\text{var}(h(X_i)+\alpha h_0(X_i))<\text{var}(h(X_i))\tag{2}$$ it does not necessarily imply that $$\text{var}\left\{\sum_{i=1}^n h(X_i)+\alpha h_0(X_i)\right\}<\sum_{i=1}^n \text{var}(h(X_i))$$ because the $h(X_i)+\alpha h_0(X_i)$'s may turn out to be positively correlated.

$\endgroup$
1
  • $\begingroup$ @Ben: I have seen this request and do not object. $\endgroup$
    – Xi'an
    Aug 31 at 14:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.