1
$\begingroup$

The population distribution is unknown, but is probably multi-modal. The required margin of error is given in advance. Any number of samples can be drawn from the population.

Given enough samples, the sample mean will converge to the population mean. How do you estimate the margin of error at each step, so the process can be stopped once the required accuracy is reached?

In case it's relevant, this arises from adaptive supersampling: determining how many rays need to be cast to converge on the correct pixel value, to a given level of accuracy.

$\endgroup$
2
$\begingroup$

If data are nearly normal, then a 95% confidence interval for the population mean is of the form $\bar X \pm t^*S/\sqrt{n},$ where $\bar X$ and $S$ are the sample mean and standard deviation, respectively, and $t^*$ cuts 2.5% of the probability from the upper tail of Student's t distribution with $\nu = n-1$ degrees of freedom.

The margin of error is $E =t^*S/\sqrt{n}$ and in most cases you will find that $n$ is large enough that $t^* \approx 2.$ Also, for moderately large $n$ $S \approx \sigma,$ the population standard deviation. So if you know the required $E$ and have a reasonable guess as to $\sigma$ then you can easily solve for $n.$

As an elementary example, heights of women in various populations have a standard deviation of about 3 inches. As a guide in ordering sizes of clothing for women, the bookstore at State U would like to have a 95% CI for the average height of women in the State U population. So we have $E = 2\sigma/\sqrt{n}$ and $n \approx (2\sigma/E])^2 = (6/.2)^2 = 144.$ So a sample of size $n = 144$ should suffice.

This formula is based on using $\bar X$ to estimate the population mean $\mu.$ Thus, if women's heights are roughly normal then $\bar X$ should be nearly normal and $t^* = 1.98$ (from R) is plenty close to $2.$ The weak link in the computation above is knowing (or guessing) the value of $\sigma$ and assuming $S \approx \sigma.$ Heights of women are nearly normal, so that would not be a difficulty in the bookstore application.

qt(.975, 143)
[1] 1.976692

However, in your application, you say the data may be bimodal and hence perhaps too far from normal for the method discussed above to be useful. For samples as large as 144 it would be unusual for this to be a serious problem. But you can use a bootstrap procedure on similar data from previous studies to check the feasibility of using the t confidence interval.

Suppose you have the following 350 bimodal data which you feel may be somewhat like the data for your proposed study. Let's see how a bootstrap would work.

set.seed(2020)
x = round(c(rnorm(150, 65, 2), rnorm(200, 71, 2)))
summary(x); sd(x)
   Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
  59.00   65.00   69.00   68.37   71.00   76.00 
[1] 3.611185
hist(x, br=20, col="skyblue2", 
     main="Existing Bimodal Data")

enter image description here

Let's see how a bootstrap procedure would work. There are many varieties of bootstrap confidence intervals. The following very elementary one (in R) should suffice for our purposes. The vector d.re, based on re-sampling with replacement from the 350 observations available, gives an idea of the variability of the sample mean.

The resulting 95% nonparametric bootstrap CI $(67.80, 68.94)$ is of length $1.14,$ which corresponds to a $0.54$ margin of error.

a.obs = mean(x)
d.re = replicate(10^3, mean(sample(x, 150, rep=T)) - a.obs)
UL = quantile(d.re, c(.975,.025))
Boot.CI = a.obs-UL;  Boot.CI
   97.5%     2.5% 
67.79810 68.93826         # 95% bootstrap CI
as.numeric(diff(Boot.CI))
[1] 1.140167              # length of bootstrap CI
| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ So if I understand correctly, the confidence interval using $E =t^*S/\sqrt{n}$ should be close enough if there are enough samples, even if the population distribution is multi-modal? $\endgroup$ – Octa9on May 31 at 17:06
  • 1
    $\begingroup$ Yes, but less predictably. And there are degrees of multi-modality not explicitly discussed here. My example for the bootstrap is just barely noticeable as bimodal in a histogram. If you have several widely separated 'modes' that will tend (a) to increase variance and req'd $n$ according to the formula, but (b) larger $n$ is better for the robustness of a t CI. // Overt muti-modality should cause you to wonder if data have messages you haven't considered. Also much of what you read is for normal data, so keep your eyes open for the unexpected. $\endgroup$ – BruceET May 31 at 17:39
  • $\begingroup$ Ok, thanks. The multi-modality of the population arises mostly from object edges, as some rays will sample the object itself, and other rays will sample whatever is behind the object. I will go ahead and accept your answer since it gives me a usable approach. $\endgroup$ – Octa9on May 31 at 18:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.