Calculating required sample size, precision of variance estimate?

Question

Background

I have a variable with an unknown distribution.

I have 500 samples, but I would like demonstrate the precision with which I can calculate variance, e.g. to argue that a sample size of 500 is sufficient. I am also interested in knowing the minimum sample size that would be required to estimate variance with a precision of $X\%$.

Questions

How can I calculate

1. the precision of my estimate of the variance given a sample size of $n=500$? of $n=N$?
2. How can I calculate the minimum number of samples required to estimate the variance with a precision of $X$?

Example

Figure 1 density estimate of the parameter based on the 500 samples.

Figure 2 Here is a plot of sample size on the x-axis vs. estimates of variance on the y axis that I have calculated using subsamples from the sample of 500. The idea is that the estimates will converge to the true variance as n increases.

However, the estimates are not valid independent since the samples used to estimate variance for $n \in [10,125,250,500]$ are not independent of each other or of the samples used to calculate variance at $n\in [20,40,80]$

Answer 1

For i.i.d. random variables $X_1, \dotsc, X_n$, the unbiased estimator for the variance $s^2$ (the one with denominator $n-1$) has variance:

$$\mathrm{Var}(s^2) = \sigma^4 \left(\frac{2}{n-1} + \frac{\kappa}{n}\right)$$

where $\kappa$ is the excess kurtosis of the distribution (reference: Wikipedia). So now you need to estimate the kurtosis of your distribution as well. You can use a quantity sometimes described as $\gamma_2$ (also from Wikipedia):

$$\gamma_2 = \frac{\mu_4}{\sigma_4} - 3$$

I would assume that if you use $s$ as an estimate for $\sigma$ and $\gamma_2$ as an estimate for $\kappa$, that you get a reasonable estimate for $\mathrm{Var}(s^2)$, although I don't see a guarantee that it is unbiased. See if it matches with the variance among the subsets of your 500 data points reasonably, and if it does don't worry about it anymore :)

Answer 2

Learning a variance is hard.

It takes a (perhaps surprisingly) large number of samples to estimate a variance well in many cases. Below, I'll show the development for the "canonical" case of an i.i.d. normal sample.

Suppose $Y_i$, $i=1,\ldots,n$ are independent $\mathcal{N}(\mu, \sigma^2)$ random variables. We seek a $100(1-\alpha)\%$ confidence interval for the variance such that the width of the interval is $\rho s^2$, i.e., the width is $100\rho \%$ of the point estimate. For example, if $\rho = 1/2$, then the width of the CI is half the value of the point estimate, e.g., if $s^2 = 10$, then the CI would be something like $(8,\,13)$, having a width of 5. Note the asymmetry around the point estimate, as well. ($s^2$ is the unbiased estimator for the variance.)

"The" (rather, "a") confidence interval for $s^2$ is $$ \frac{(n-1) s^2}{\chi_{(n-1)}^{2\;(1-\alpha/2)}} \leq \sigma^2 \leq \frac{(n-1) s^2}{\chi_{(n-1)}^{2\;(\alpha/2)}} \>, $$ where $\chi_{(n-1)}^{2\;\beta}$ is the $\beta$ quantile of the chi-squared distribution with $n-1$ degrees of freedom. (This arises from the fact that $(n-1)s^2/\sigma^2$ is a pivotal quantity in a Gaussian setting.)

We want to minimize the width so that $$ L(n) = \frac{(n-1) s^2}{\chi_{(n-1)}^{2\;(\alpha/2)}} - \frac{(n-1) s^2}{\chi_{(n-1)}^{2\;(1-\alpha/2)}} < \rho s^2 \>, $$ so we are left to solve for $n$ such that $$ (n-1) \left(\frac{1}{\chi_{(n-1)}^{2\;(\alpha/2)}} - \frac{1}{\chi_{(n-1)}^{2\;(1-\alpha/2)}} \right) < \rho . $$

For the case of a 99% confidence interval, we get $n = 65$ for $\rho = 1$ and $n = 5321$ for $\rho = 0.1$. This last case yields an interval that is (still!) 10% as large as the point estimate of the variance.

If your chosen confidence level is less than 99%, then the same width interval will be obtained for a lower value of $n$. But, $n$ may still may be larger than you would have guessed.

A plot of the sample size $n$ versus the proportional width $\rho$ shows something that looks asymptotically linear on a log-log scale; in other words, a power-law--like relationship. We can estimate the power of this power-law relationship (crudely) as

$$ \hat{\alpha} \approx \frac{\log 0.1 - \log 1}{\log 5321 - \log 65} = \frac{-\log 10}{\log \frac{5231}{65}} \approx -0.525 , $$

which is, unfortunately, decidedly slow!

---

This is sort of the "canonical" case to give you a feel for how to go about the calculation. Based on your plots, your data don't look particularly normal; in particular, there is what appears to be noticeable skewness.

But, this should give you a ballpark idea of what to expect. Note that to answer your second question above, it is necessary to fix some confidence level first, which I've set to 99% in the development above for demonstration purposes.

Answer 3

I would focus on the SD rather than the variance, since it's on a scale that is more easily interpreted.

People do sometimes look at confidence intervals for SDs or variances, but the focus is generally on means.

The results you give for the distribution of $s^2/\sigma^2$ can be used to get a confidence interval for $\sigma^2$ (and so also $\sigma$); most introductory math/stat texts would give the details in the same section in which the ditribution of $\sigma^2$ was mentioned. I would just take 2.5% from each tail.

Answer 4

The following solution was given by Greenwood and Sandomire in a 1950 JASA paper.

Let $X_1,\dots,X_n$ be a random sample from a $\mathrm{N}(\mu,\sigma^2)$ distribution. You will make inferences about $\sigma$ using as (biased) estimator the sample standard deviation $$ S=\sqrt{\sum_{i=1}^n\frac{(X_i-\bar{X})^2}{n-1}}, $$ and you want to control the probability that the relative deviation between $S$ and $\sigma$ is within a fraction $0<u<1$. That is, $$ \Pr\{S<(1-u)\cdot\sigma\}=a \quad\text{and}\quad \Pr\{S>(1+u)\cdot\sigma\}=b, $$ in which the significance level $\gamma=1-a-b$.

It follows that $$ \Pr\!\left\{ \frac{(n-1)S^2}{\sigma^2} < (n-1)(1-u)^2\right\} = a $$ and $$ \Pr\!\left\{ \frac{(n-1)S^2}{\sigma^2} > (n-1)(1+u)^2\right\} = b. $$ Since the pivotal quantity $(n-1)S^2/\sigma^2$ has $\chi^2_{n-1}$ distribution, adding the two probabilities, we find

$$ \gamma = F_{\chi^2_{(n-1)}}((n-1)(1+u)^2) - F_{\chi^2_{(n-1)}}((n-1)(1-u)^2), $$

and the necessary sample size is found solving the former equation in $n$ for given $\gamma$ and $u$.

R code.

gamma <- 0.95
u <- 0.1
g <- function(n) pchisq((n-1)*(1+u)^2, df = n-1) - pchisq((n-1)*(1-u)^2, df = n-1) - gamma
cat("Sample size n = ", ceiling(uniroot(g, interval = c(2, 10^6))$root), "\n")

Output for $u=10\%$ and $\gamma=95\%$.

Sample size n = 193