3
$\begingroup$

I'm working on an experiment to measure property $X$ of some pieces of equipment. Assuming there is an excess of equipment available for testing, and barring any funding restrictions, what is the best procedure for choosing how many measurements to make, so that I have a set confidence interval width?

Valid values for property $X$ are strictly positive, but their range is quite wide; typical values can reasonably be as little as $0.01$ and as large as $3000$. For this particular test, I expect the measurements to fall in the $3-20$ range.

Improve this question
$\endgroup$

2 Answers 2

Reset to default
2
$\begingroup$

In my experience, taking a small sample (less than 10, or 5% of the overall number of equipments) doing several measurements gives an indication of how many measurements need to be done to have a good CI.

For instance, if I have 50 pieces, I am going to take 5 pieces, do several measurements until I find a balance for each of the 5 pieces, ranging from 3 measurements to 10 first. If I don't find a balance, I'll redo the measurements, but this time 11 to 20 times (and so on). Once I have a smoothed average of measurements, I'll deploy the method to the overall pieces.

This will give you an estimated standard deviation.

Based on that and your level of confidence, you can, then, build your CI with the formula I assume you already know.

Improve this answer
$\endgroup$
5
  • $\begingroup$ That helps, thanks. In my case, though, the individual measurements destroys the piece of equipment, so I can't really do more than one measurement on each piece. Also, I assume this initial procedure is to get a feel for the sample standard deviation, correct? What should I do with it subsequently? $\endgroup$
    – LmnICE
    Aug 20, 2015 at 11:22
  • $\begingroup$ Yes, it is. Once you have an estimation of your standard deviation, you can choose the level of confidence of your interval (85, 90, 95, 99%). Then you can calculate the CI, check if it falls within your expected range. - How many measurement can you do? $\endgroup$
    – Amandine FAURILLOU
    Aug 20, 2015 at 11:43
  • $\begingroup$ I'd say it would be ideal to keep the number of measurements under 20 (seeing as each one takes a good 8 hours), but I guess it would depend on how tight I'd want the CI to be, which I don't know yet. Assuming the errors are normally distributed, I'm wondering if using a straight normal distribution for the sample mean is appropriate (as opposed to, say, a Student-t to correct for the low number of samples). $\endgroup$
    – LmnICE
    Aug 20, 2015 at 12:10
  • $\begingroup$ Given your expected range, do you have an expected mean or SD? - if the distribution is gaussian, then using a gaussian distribution seems appropriate, the low number of the sample only intervenes if the distribution is not gaussian, over 30 gaussian and student give both more or less the same intervals. I would start on 6 measurements to test the waters (so to speak) $\endgroup$
    – Amandine FAURILLOU
    Aug 20, 2015 at 12:45
  • $\begingroup$ I have an estimate of the SD, by assuming normality and that the range represents a $95\%$ CI. Since I was expecting to do something like 10-15 samples, I though that assuming normality would be folly. Anyway, that sounds like a good strategy. Thanks again! $\endgroup$
    – LmnICE
    Aug 21, 2015 at 15:27
3
$\begingroup$

I am hardly a good statistician, but I'll try to contribute. Can't comment for now (not enough Reputation) so directly as an answer. Assuming you want a CI at 95% and normal distribution of your measurements, CI definition is

$CI= 1.96* (sd /\sqrt{N})$ with sd the standard deviation and N the number of samples.

Your measurements will be between $3-20$, so you might assume worst-case-scenario with large sd. Based on thin air I assumed a sd of $15$ but very simple (and perhaps unnecessary) R simulations showed sd to be between $4$ and $6.5$ for an expected range of $3-20$. A sd of $6.5$ may then help choose a "cautious" (i.e, quite large) sample size.

EDIT: simulations for sd estimation

# Ranges for outcomes
widest_range=seq(from=0,to=3000,by=1)
likely_range=seq(from=3,to=20,by=1)

# matrix to store the results
nb_sims=30
Mat_Results=matrix(0,ncol=2,nrow=nb_sims)
colnames(Mat_Results)=c("Widest_range","Likely_range")

# sims
for (i in (1:nb_sims)) {
  # sampling assuming 10 measures (EDIT: uniform distribution)
  measures_widest=sample(widest_range, 10, replace = FALSE, prob = NULL)
  measures_likely=sample(likely_range, 10, replace = FALSE, prob = NULL)
  # sd
  Mat_Results[i,1]=sd(measures_widest)
  Mat_Results[i,2]=sd(measures_likely)
}

x11()
par(mfrow=c(1,2))
boxplot(Mat_Results[,1],main="SD with widest range")
boxplot(Mat_Results[,2],main="SD with likely range")
Improve this answer
$\endgroup$
8
  • 1
    $\begingroup$ Hey, thanks! Assuming normal distribution for my measurements and a $95\%$ CI, a range of $3-20$ for them would yield an SD of about $5$, no? $\endgroup$
    – LmnICE
    Aug 21, 2015 at 10:13
  • $\begingroup$ Yes, you're right. I don't know whether there's a mathemical rule behind that, but I tried a very quick simulation on R and I get a mean sd of $5.5$ out of 30 simulations (for $n=10$ measurements per simulation). $95$% simulated sd between $4$ and $6.5$. For the range of $0$ to $3000$ (I simplified the max. range that you were talking about), mean sd of $850$! $\endgroup$
    – NOTM
    Aug 21, 2015 at 10:50
  • $\begingroup$ That $0-3000$ range refers to acceptable values for this particular property (for any pieces of equipment). Measurement will narrow that down considerably, I hope. $\endgroup$
    – LmnICE
    Aug 21, 2015 at 10:54
  • $\begingroup$ I hope so too. I just did it for fun. Within your likely range of values, highest sd to be expected seems $6.5$. You could determine your sample size based on that value (kind of an expect-the-worse approach). $\endgroup$
    – NOTM
    Aug 21, 2015 at 11:00
  • 1
    $\begingroup$ Definitely not standard, but I don't mind! Again assuming normality, the mean of a sample whose range is $3-20$ is $11.5$. Assuming that range represents a $95\%$ CI (which is more of a leap than the normality assumption), $\sigma_{sample} = \frac{20-11.5}{1.96} = 4.34$. $\endgroup$
    – LmnICE
    Aug 21, 2015 at 11:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.