How do you experimentally determine a minimum sample size to characterize a response? I want to repeatedly perform a measurement until the distribution of response values stabilizes.  Is there a standard metric to stistically conclude that new data does not contain new information?  I want to identify when I have a sufficient number of samples to characterize the response distribution.
My ad hoc approach was to calculate the mean and standard deviation after each new sample.  With each new measurement, these values eventually stabilize to a consistent value.  I can easily check the size of the change for each step, then put a cuttoff when a change is sufficiently small.  Perhaps I could stop measuring after the change in standard deviation is less than 5% of the mean 3 measurements in a row.
Is there a more formal approach to experimentally determining a sufficient sample size? 
Here is an plot of a mock experiment where samples are randomly chosen between 0 and 1.  This illustrates how the mean and standard deviation level off as more samples are obtained.

 A: One approach is to calculate the confidence interval of the mean value after each sample.  The relative size of this interval compared to the mean value could be used to identify when to stop measuring.
The calculation begins by calculating the standard deviation of the sample, $s$, and then the standard error of the mean,  $s_{\bar x}=s/\sqrt{n}$, where $n$ is the number of samples.  The confidence interval is defined as $$\bar x \pm t s_{\bar x}$$ where $t$ is obtained from a t-distribution table, based on the confidence level and degrees of freedom.  Using the t-value in this situation is more appropriate because of the small sample size.  Note that if you specify a 95% confidence level, you would use a 1-sided t-table with $\alpha=0.5$.
It seems reasonable to set a minimum number of samples to obtain, then stop measuring after the confidence interval is repeatedly less than a prescribed percentage of the mean.  For example, stopping after the confidence interval is less than 5% of the mean, $2ts_{\bar x} < 0.05 \bar{x}$, 3 samples in a row.  
Several online references (see below) discuss identifying the required sample size based on an estimated standard deviation of the sample.  The approach here instead calculates the sample standard deviation as you go. The cuttoff criteria is somewhat arbitrary but the method for calculating the confidence interval is statistically sound.


*

*Western Michigan STAT 216: Determining Sample Size for Estimating the Mean

*PennState STAT 506: Selection Sample Size for Estimating Population Mean and Total

*Select Statistical Services: Population Mean - Sample Size

*R Tutorial: Sampling Size of Population Mean
