# Error bars on sample variance

I'm evaluating the performance of a statistical estimator under a number of parameter settings. The estimator is unbiased for all of the parameters, so I'm reporting the sample variance as a measure of quality, that is, If I am interested in approximating: $$\int h(x)\pi(x)dx$$ Where $h(x) : \mathbb{R}^d \rightarrow \mathbb{R}$ and $\pi$ is a normalized distribution over $\mathbb{R}^d$. And I am using importance sampling with proposal distribution $q$ which gives samples $\{x_1, ..., x_n\}$. I have the estimator: $$\bar{h} = \frac{1}{n}\sum_{i=1}^n h(x_i)\pi(x_i)/q(x_i)$$ The variance of $\bar{h}$ is given by: $$\frac{1}{n}\mbox{var}(h(x)\pi(x)/q(x))$$ I can approximate this variance with a test run of $m$ samples, and compute the sample variance: $$\bar{v} = \frac{1}{m-1}\sum_{i=1}^m (h(x_i)\pi(x_i)/q(x_i)-\bar{h})^2$$ Gives me the approximated value for the variance of my estimator for any number of samples $n$: $$\mathcal{V} = \frac{1}{n}\bar{v}$$ Assuming I have not done anything stupid so far, if I'm reporting $\mathcal{V}$ as a way of judging the estimator I should but error bars on it, and I'm not sure how to do it.

edit: Actually this report is very useful, they show: $$\mbox{var}(\bar{v}) = \frac{1}{m}(\mu_4 - \mu_2^2) + \mathcal{O}(m^{-2})$$ Where $\mu_k$ is the $k$-th moment of the RV, so I assume I can just approximate these moments and plug them into this formula.

-
the variance of $\bar{h}$ is dependant on the number of samples you take, but it's producted with a constant quantity, which is the variance of a single sample from the estimator, which is what $\bar{v}$ approximates –  anonymous_4322 May 31 '12 at 17:40
I still do not get the role of $m$. –  user10525 May 31 '12 at 17:55
it's an arbitrary constant of little consequence, I mean to use it to answer the question of "If I had computational budget of $m$ samples how well does this estimator do", it ended up there because some of my methods actually have higher "cost", a detail I left out. –  anonymous_4322 May 31 '12 at 18:09
Thanks for the clarification. I believe that if $n$ is large enough (which should be the case), the variance is going to be quite small (this follows from the expression for the variance you present). If $\mbox{var}(h(x)p(x)/q(x))$ is not reasonably small, this indicates that the importance function $q$ is not really efficient, I think. –  user10525 May 31 '12 at 18:15
@anonymous_4322 I am quite sure Procrastinator was referrring to the variance of the sample variance estimate and not the sample variance itself. –  Michael Chernick May 31 '12 at 19:56