1
$\begingroup$

I have a data set - results (measurements) of experiments for several years. Now I want to reduce the number of experiments (to save the money, time etc.) without a big loss in precision and power.

Which statistical test can be applied to check, if the results of experiments are still reliable/accurate/precise by conducting not 100%, but 90% of the original number of experiments? Can I take 90%-bootstrap samples from my data set and compare them with original data set, for example by means KS-test?

Improve this question
$\endgroup$
3
  • $\begingroup$ Why bootstrap? You mean you want to randomly sample with replacement each single data entry until you get a sample size totalling 90% of the original one? I would avoid replacement. In any case you might use a paired sample t test. $\endgroup$
    – Giuseppe Biondi-Zoccai
    Feb 2, 2017 at 16:43
  • $\begingroup$ Yes, this is want I meant. Roughly speaking, I have a sample of 100 individuals (as example), and a certain value (weight, length, blood presure, etc.) of each individual is measured, to estimate this value for the population. The main question is: would be (approximately) the same level of precision provided, if we decide to save the time and costs and sample 90 individuals instead of 100? $\endgroup$
    – J. Wish
    Feb 2, 2017 at 21:43
  • 2
    $\begingroup$ You are right, I want to generate a large number of samples consisting of 90 elements from original 100 ones, with replacement (well, it's not really bootstraping...). Why would you avoid replacement? Thank you, I will try a paired t-test. $\endgroup$
    – J. Wish
    Feb 2, 2017 at 21:54

1 Answer 1

Reset to default
1
$\begingroup$

I cannot see why this is a problem of hypothesis testing, it looks more like experimental design. But, from information in comments it looks like you are interested in estimating population means. If so, and assuming population variance isn't changing (much) with time, you can simply use $$ \mathbb{Var}(\bar{X}_n) =\sigma^2 / n $$ where $\sigma^2$ is the population variance, and $n$ sample size. Then you get an answer without any simulation.

For somewhat more complicated cases, you can simply draw without replacement (not bootstrap) from historical sample for various sample sizes lesser than $n$, and compare.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.