I’m currently reading “Improving the sensitivity of online controlled experiments by utilizing pre-experiment data” by Deng et al. and struggling to derive a few equations from the paper. I would appreciate some help from those who are familiar with the paper.

The first problem is with the first equation on page 4:

$$ \text{var}(\hat{Y}_\text{strat}) = \sum_{k = 1}^K \frac{w_k}{n} \sigma^2_k. $$

It does not seem to match the one given on Wikipedia:

$$ s_{\bar {x}}^{2}=\sum _{h=1}^{L}\left({\frac {N_{h}}{N}}\right)^{2}{\frac {s_{h}^{2}}{n_{h}}}. $$

Presumably, $w_k = N_h/N$ and $\sigma^2_k = s_h^2/n_h$, but then $w_k$ is not squared, and the equation has an extra $n$ in the denominator.

The second problem is with equation 5 on the same page. How does one substitute

$$ \theta = \frac{\text{cov}(X, Y)}{\text{var}(X)} $$


$$ \text{var}(\hat{Y}_\text{cv}) = \frac{1}{n}(\text{var}(Y) + \theta^2 \text{var}(X) - 2 \theta \text{cov}(X, Y)) $$

to get

$$ \text{var}(\hat{Y}_\text{cv}) = \text{var}(\bar{Y})(1 - \rho^2)? $$

A sheet of paper and a pencil do not seem to help. Thank you!

  • $\begingroup$ First question: you were probably missing the definition of $n_k$ before equation (2). $\endgroup$ – Yair Daon Nov 27 '19 at 20:55

First question:

$ \begin{align} \begin{split} Var(\hat{Y}_{strat}) &= Var(\sum_k w_k \bar{Y}_k) \\ &=\sum_k w_k^2 Var(\frac{1}{n_k}\sum Y^{(k)}_i) \\ &=\sum_k w_k^2 \frac{1}{n_k^2} n_k Var(Y^{(k)}_1)\\ &=\sum_k w_k^2 \frac{1}{n_k}\sigma_k^2\\ &=\sum w_k^2 \frac{1}{n w_k} \sigma_k^2\\ &=\sum w_k \frac{\sigma_k^2}{n}\\ \end{split} \end{align} $

You were probably missing the definition of $n_k$ above equation (2).

Second question:

$ \begin{align} \begin{split} Var(Y) + \theta^2 Var(X) -2\theta Cov(X,Y) &= Var(Y) + \frac{Cov^2(X,Y)}{Var^2(X)} Var(X) - 2\frac{Cov(X,Y)}{Var(X)} Cov(X,Y)\\ &=Var(Y) - \frac{Cov^2(X,Y)}{Var(X)}\\ &=Var(Y)(1-\frac{Cov^2(X,Y)}{Var(X)Var(Y)})\\ &=Var(Y)(1-Corr^2(X,Y)) \end{split} \end{align} $

Then divide by $n$ to get the variance of the sample mean.

| cite | improve this answer | |
  • $\begingroup$ Right! In the first, I was indeed missing $n_k = w_k n$, and in the second, I actually got it, but for some reason, I failed to see it was correct. Thank you! $\endgroup$ – Ivan Nov 28 '19 at 6:11
  • $\begingroup$ You're welcome. Vote up and give me the bounty please. $\endgroup$ – Yair Daon Nov 28 '19 at 8:18
  • $\begingroup$ Sure! It’s just not possible within 24 since the bounty start. $\endgroup$ – Ivan Nov 28 '19 at 11:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.