What is the variance of this estimator

Question

I want to estimate the mean of a function f, i.e. $$E_{X,Y}[f(X,Y)]$$ where $X$ and $Y$ are independent random variables. I have samples of f but not iid: There are iid samples for $Y_1,Y_2,\dots Y_n$ and for each $Y_i$ there are $n_i$ samples from $X$: $X_{i,1},X_{i,2},\dots, X_{i,n_i}$

So in total I have samples $f(X_{1,1},Y_1) \dots f(X_{1,n_1},Y_1 ) \dots f(X_{i,j},Y_i) \dots f(X_{n,n_n},Y_n)$

To estimate the mean I compute $$\mu=\sum_{i=1}^n 1/n * \sum_{j=1}^{n_i}\frac{ f(X_{i,j},Y_i)}{n_i}$$ Obviously $$E_{X,Y}[\mu]=E_{X,Y}[f(X,Y)]$$ so $\mu$ is an unbiased estimator. I am wondering now what $Var(\mu)$, i.e. the variance of the estimator is.

Edit 2: Is this the correct variance? $$Var(\mu)=\frac{Var_Y(\mu_i)}{n}+\sum_{i=1}^n \frac{Var_X(f(X,Y_i)))}{n_i*n^2}$$ It seems to work in the limit, i.e. if n=1 and all $n_i=\infty$ the variance just becomes the variance of the means. And if $n_i=1$ the formula becomes the standard formula for the variance of estimators. Is this correct? How can I proof that it is?

Edit (Ignore this):

So I think I made some progress: Let us first define $\mu_i=\sum_{j=1}^{n_i}\frac{ f(X_{i,j},Y_i)}{n_i}$ which is an unbiased estimator of $E_X[f(X,Y_i)]$.

Using the standard formula for variance we can write:

$$Var(\mu)=1/n^2 \sum_{l=1}^n \sum_{k=1}^n Cov(\mu_l,\mu_k)$$ This can be simplified to $$1/n^2( \sum_{i=1}^n Var(\mu_l)+ 1/n^2\sum_{l=1}^n \sum_{k=l+1}^n 2*Cov(\mu_l,\mu_k))$$ and because the $X_{ij}$s are drawn independently we can further simplify this to $$1/n^2( \sum_{i=1}^n 1/n_i Var(f(X_{i,j},Y_i))+1/n^2 \sum_{l=1}^n \sum_{k=l+1}^n 2*Cov(\mu_l,\mu_k))$$ And for the covariance: \begin{align} Cov(\mu_l,\mu_k)&=Cov(\sum_{j=1}^{n_l} \frac{f(X_{j,l},Y_l)}{n_{l}},\sum_{j=1}^{n_k} \frac{f(X_{j,k},Y_k)}{n_{k}})\\ &=\frac{1}{(n_k*n_l)}*Cov(\sum_{j=1}^{n_l} f(X_{j,l},Y_l),\sum_{j=1}^{n_k} f(X_{j,k},Y_k))\\ &=\frac{1}{(n_k*n_l)}*\sum_{j=1}^{n_l}\sum_{j=1}^{n_k}Cov( f(X,Y_l), f(X,Y_k))\\ &=\frac{n_k*n_l}{(n_k*n_l)}Cov( f(X_{i,l},Y_l), f(X_{i,k},Y_k))\\ &=Cov( f(X,Y_l), f(X,Y_k)) \end{align} So plugging this back in we get $$1/n^2( \sum_{i=1}^n 1/n_i Var(f(X,Y_i))+1/n^2 \sum_{l=1}^n \sum_{k=l+1}^n 2*Cov(f(X,Y_l),f(X,Y_k)))$$ I have multiple questions now:

1. Is the calculation above correct?

2. How can I estimate $Cov(f(X,Y_l),f(X,Y_k)))$ from the samples given?

3. Does the variance converge to 0 if I let n go to infinity?

Answer 1

Q1: No, it's not quite right. You omit the subscripts in line 3 of your final derivation of the covariance. That obscures the fact that the two RVs labeled "X" are in fact independent of one another: one had an $\ell$ subscript and the other a $k$. In that whole block of equalities, the only nonzero terms should be when $k=\ell$, because functions of independent inputs are independent. (I assume you are okay with saying $X_{12}, Y_1$ is independent of $X_{22}, Y_2$ even though this does not follow, strictly speaking, from pairwise claims of independence among all the $X$'s and $Y$'s.)

Q2: From above, that term is nonzero only when $k=\ell$, and in that case, it reduces to $Cov(f(X_{jk}, Y_k), f(X_{jk}, Y_k)) = Var(f(X_{jk}, Y_k))$. The result after the sum is $Cov(\mu_k, \mu_k) = \frac{1}{n_k} Var(f(X_{jk}, Y_k))$.

Q3: Yes: after these modifications, you will have only a linear number of terms in the very last sum, so the denominator's quadratic term will win.