# The fallacy of splitting data collection time into shorter intervals to reduce error

Say you measure the (roughly constant) rate of something (i.e., counts of something per unit time) over 30 seconds, and you repeat this reading four times to find the mean and standard error of the mean.

Why does splitting this up into 12 repeats of a 10 second reading not reduce the standard error of the mean rate? I mean, one would naively think that because there are more repeated measurements, the standard error of the mean would decrease.

• By "rate of something" would you mean a count of events per unit time? If not, then please tell us more about this rate and how it is measured.
– whuber
Mar 18, 2015 at 18:47
• @whuber Yes, counts per unit time. I'm trying to consider a general case here. Mar 18, 2015 at 18:56
• Why not clarify your other question? Aside from talking about time rather than distance, is this really a fundamentally different question? Mar 19, 2015 at 2:01

A constant rate of events per unit time is called a Poisson process when the outcomes in one time interval are independent of the outcomes in any other time interval. This independence assumption is usually valid and supported by physical considerations. When not, it can be tested.

A Poisson process is characterized by a single parameter. The rate per unit time is usually chosen. Let's call it $\lambda$. This means the expected number of events observed throughout a duration $dt$ is $\lambda dt$. The variance of the number of events is also equal to $\lambda dt$.

Suppose you were to split the duration $dt$ into nonoverlapping subintervals $dt_1, dt_2, \ldots, dt_k$, with $dt_1 + dt_2 + \cdots + dt_k = dt$. Then:

• The expected number of observations would equal the sum of the expectations,

$$\sum_{i=1}^k \lambda dt_i = \lambda \sum_{i=1}^k dt_i = \lambda dt,$$

just as it should.

• The variance of the number of observations would equal the sum of the variances because the numbers of observations are independent. The calculation is exactly the same, only this time the quantities represent the variances:

$$\sum_{i=1}^k \lambda dt_i = \lambda \sum_{i=1}^k dt_i = \lambda dt.$$

Therefore, splitting the data into groups of observations does not improve the precision with which you can estimate the rate.

It can pay to go a little further. Suppose the rate does vary over time, but slowly enough that assuming a constant rate within each subinterval $dt_i$ is a good approximation. Let $X_i$ have Poisson$(\lambda_i)$ distributions and be independent, just as before. (To simplify the notation, I have incorporated the dependence on the durations $dt_i$ within the parameters $\lambda_i$.) Upon observing each of the $X_i$ you might want to estimate the underlying mean rate (per smaller interval) as the sample mean

$$\hat\lambda = \frac{1}{k} \sum_{i=1}^k x_i$$

and the variance in that rate as the sample variance

$$\hat\sigma^2 = \frac{1}{k-1} \sum_{i=1}^k (x_i - \hat\lambda)^2.$$

Using the Poisson distribution facts that $\mathbb{E}(X_i) = \lambda_i$ and $\mathbb{E}(X_i^2) = \text{Var}{X_i} + \mathbb{E}(X_i)^2 = \lambda_i + \lambda_i^2$, a little algebra shows that

$$\mathbb{E}(\hat\lambda) = \frac{\lambda}{k}$$

($\lambda = \sum_i \lambda_i$), which we may interpret as the mean rate per (smaller) interval, and

$$\mathbb{E}(\sigma^2) = \frac{1}{k-1}\left(\sum_i (\lambda_i - \lambda/k)^2\right) + \frac{\lambda}{k}.$$

The right hand term is the variance of a Poisson process of constant rate $\lambda/k$. The other term is the variance of the separate rates. When there is no variation, the first term drops out and this reduces to the previous result: the variance equals the mean and from that we derive the standard error as usual. But when the underlying rate does vary, its variance contributes to the expected sample variance, creating an overdispersed dataset.

Incidentally, the effect of the subdivision is never to decrease the estimated variance (and therefore the standard error): it can only increase it.

We see from this that the potential advantage of dividing the whole time interval into little parts is that it allows us to detect variations in the underlying rate and to use them to increase our estimate of the standard error. If you are confident that the underlying rate does not appreciably change, then there is no need to collect data from subintervals--but it couldn't hurt.

One easy way to see that this is that in both cases, when asked to predict the mean you're going to predict the same thing. If your rate is given by $f(x)$,your first four readings are equal to $$x_i =1/30 \int_{30i}^{30i + 30}f(x)dx$$ for $i \in \{0,1,2,3 \}$ and your other readings are $$y_j =1/10 \int_{10j}^{10j + 10}f(x)dx$$ for $j \in \{0,1,\dots, 11\}$, then in both cases your prediction for the mean will be equal, $\sum_i x_i/4 = \sum_j y_j/12$. Therefore neither can be closer to the true mean. Now what is wrong with the naive logic that you presented?

While we have more samples, they are from a different distribution, one with greater variance, and these effects cancel out completely.

One can construct a similar example. Let $x_i \sim \mathcal{N}(0,1)$ for $i=1, \dots 2n$. Now define $y_j = x_{2j} + x_{2j+1}$. Basic properties of normal distributions will confirm that $y_j \sim \mathcal{N(0, 1/2)}$. And by the same logic

$$\sum_i x_i/2n = \sum_j y_j/n \sim \mathcal{N(0,1/2n)}.$$

So the sample mean of both distributions is an equally accurate estimate of the mean. With $x_i$ we have more samples, but greater variance.

• Whilst I understand the gist of the answer, when I try to calculate the standard error of the mean for each case, I get equal SDs and an SEM that is $\frac{1}{\sqrt{2}}$ smaller for the smaller-and-more-intervals case. I've tried to calculate these by plugging $2N$ in place of $N$, and $\frac{1}{2}x_i$ in place of $x_i$ in the formulae for SD and SEM Mar 18, 2015 at 19:26