It is all easy and intuitive when the experimental unit is discrete, "a person", "a mouse" etc. However, say Peter has taken 5 soil samples from each experimental plot (which is the experimental unit), running a protocol of minuscule analytical measurement 5 times with the 5 samples, each time 0.1g of soil is used (n=5 at the plot level). Tom did it another way, he dug 5 samples and composited them together and run a larger (more wasteful) analytical protocol taking 10g of sample at once and only did the measurement once (n=1 at the plot level). Assuming both analytical protocol is equally valid, Peter as a whole analyzed 0.5g, but Tom analyzed 10g of soil! However, when you run statistical analysis and compare different plots, Tom's result would be less powerful because of the smaller sample size (number), right?

Surely, there has to be some way to "formalize" a continuous experimental unit, like every gram of soil per m2 of soil * 10cm deep is considered a sample size of 1. But it seems nobody has proposed anything close to it in soil science. Am I missing something? Did I mix up something between statistical power and representativeness? (which the latter is more subjective and can't be quantified)

Edit: Let's simplify this, say there are only two plots and we are gonna compare them by t-test. Peter has 5 measurements (each measurement used only 0.1g) from each plot and Tom has only 2 measurements per plot but he used a protocol which process more soil i.e. 10g per measurement (you cant even do a legit t-test if you have 1 measurement each plot). Also, note that a perfect homogenization of soil samples is very unlikely, with some soil properties even susceptible to change from homogenization efforts. Surely there are some logical problems when we take the conventional meaning of sample "size", no?


All sample size calculations are built on top of a proposed inference that will be made from the data. This might be a confidence interval for an unknown parameter, or a hypothesis test for a set of hypotheses, or a Bayesian posterior inference, etc. Whatever the inference being made, there will be some appropriate measure of how "accurate" the inference is, and this accuracy will be a function of the sample size. For example, if you are computing a confidence interval then the accuracy is usually measured by the width of the interval (relative to an unknown standard deviation) at a given confidence level.

If you want to formalise a sample size calculation for a continuous measure of size (e.g., the weight of the sampled soil), you will need to formulate the inference that is being made with the sample, and write out the accuracy of the inference as a function of the continuous size measure. So long as you can write out the accuracy of the proposed inference as a function of the size, you can determine the minimum size required to obtain a stipulated minimum level of accuracy. This can be done regardless of whether the sample size is specified by a discrete unit or a continuous measure.

Example: Suppose you have an experiment where you will sample a weight of $w$ grams of soil and determine the proportion of some mineral in that sample. Suppose you are willing to stipulate that the sample proportion $p$ is related to the true proportion $\theta$ by the sampling distribution:

$$p \sim \text{N} \Bigg( \theta, \frac{\theta (1-\theta)}{w} \Bigg).$$

In this case you might make an inference about the true proportion $\theta$ using the confidence interval:

$$\text{CI}_\theta(1-\alpha) = \Bigg[ p \pm z_{\alpha/2} \sqrt{\frac{p (1-p)}{w}} \Bigg].$$

The length of this confidence interval is:

$$L= 2 z_{\alpha/2} \sqrt{\frac{p (1-p)}{w}}.$$

A higher accuracy for a confidence interval requires a smaller length for the interval (i.e., a narrower interval is a more accurate inference). Thus, to stipulate the minimum required accuracy for our inference, we would stipulate some maximum length $L_*$ that we are willing to accept. For a given value $\alpha$ and a given sample proportion $p$, getting this stipulated length requires us to set the sample weight to:

$$w = 4 z_{\alpha/2}^2 \frac{p (1-p)}{L_*^2}.$$

Note that this formula will generally yield a non-integer value, which is okay in the case where our sample weight is continuous. As you can see, there is nothing fundamentally different in this computation to the case where we have a discrete sample size. (The only difference here is that we do not need to round the required sample size up to the next integer at the end of the computation.) What is needed is for us to be able to write out some measure of the accuracy of the inference as a function of the sample weight, and then find the minimum weight that gives some stipulated minimum accuracy.

  • $\begingroup$ Thanks for your answer. Let me digest it for some time. $\endgroup$ – y chung Apr 13 '20 at 16:11

Peter cannot inflate his effective sample size for the purpose of estimating treatment effects just by repeatedly subsampling the same experimental units -- this would be the most egregious form of 'pseudoreplication'.

Sample size in the context of a designed experiment is set by the randomization design -- since the different samples within the same plot could not possibly be assigned to receive different treatments, they cannot be considered separate experimental units. It doesn't matter, for the purpose of sample size, how much soil Tom or Peter collect from each unit -- what matters is that all the soil in that unit received the same treatment -- that all of it was required by the experimental design to receive the same treatment.

Tom and Peter can potentially take different measurements of the same experimental unit, and one might get "better" measurements in some sense than the other -- maybe measurement error is reduced by using a larger volume of soil, or maybe it's reduced by averaging samples from several points within the plot -- but that's an issue of reducing the size of the error variance (assumed the same for each plot) not inflating the sample size.

A more precise/reliable/stable measurement method could thereby still reduce the standard errors of effect estimates, but not through changing the sample size. The sample size is, again, fixed by the design of the randomization scheme -- each unit that could be independently assigned to a different treatment is one experimental unit and adds one to the total sample size.

  • $\begingroup$ Thanks for your answer. I think nested ANOVA already took care of subsampling, no? $\endgroup$ – y chung Apr 13 '20 at 16:11

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.