Standard deviation vs standard error of the mean for intervals

Question

I couldn't find a question like this anywhere on Cross Validated. Also I struggled to write the title for this question.

Imagine I have a satellite picture of a straight road, and I expect that the number of cars per unit length is roughly constant. I choose a section of the road, and split it into $N=10$ equal intervals. I then find the number of cars per unit length for each interval, $x_i$ for $i$ in range 1 to 10.

I then calculate the mean number of cars per unit length

$\hat{\mu} = \bar{x} = \sum\limits_{i=1}^N \frac{x_i}{N}$,

and the sample standard deviation

$\hat{\sigma} = s = \sqrt{\frac{N}{N-1} (\sum\limits_{i=1}^N \frac{x_i^2}{N}-\hat{\mu}^2)}$

I want a value for the error in the mean number of cars per unit length. So, I think, I must find the estimated standard error of the mean, $\frac{s}{\sqrt{N}}$.

However, does this not mean that if I shrink the length of my intervals whilst increasing the number of intervals $N$ so that it still covers the same section of the road, I will get a lower standard error of the mean? Isn't this reduction of the error value cheating? It leads me to believe that I should use the standard deviation as my error value.

But what if I increase the range of my section, so that I genuinely have a higher $N$? I do expect to have a lower error in that case.

Which should I use, and have I made a mistake in my thinking somewhere?

Answer 1

Note that you defined your quantities to be the mean and standard deviation of cars per unit length.

So let's imagine your section is $k$ units long, and you take $n=k$ intervals.

Then your $s$ really does estimate standard deviation of counts-per-unit-length.

Now take $n=2k$ intervals. Is the standard deviation you get from counts in intervals of length $\frac{1}{2}$ the standard deviation per unit length? No, it's the standard deviation of counts per half-unit length.

Those will on average be smaller. They'll be multiplied in effect by a factor of $\sqrt{\frac{1}{2}}$.

So when you now divide $s$ by $\sqrt{n}=\sqrt{2k}$, you're computing the standard error of the mean-per-half-unit-length, which is not what you wanted at all. Of course it gets smaller when you do that. You'll have a number that's on average half as big as the thing you were trying to work out.

So when you scale back to the equivalent per unit length, you have to double the mean to get a mean per unit length, and correspondingly, you will have to similarly scale the standard error for the rescaling of the mean (doubling it). There's actually no gain at all (the only difference comes from the small effect of Bessel's correction - the population quantities being estimated scale exactly).

Answer 2

Your problem description sounds like a Poisson-type scenario. I don't see any issue with how you estimate the number of cars per unit length. Let's take an example.

Suppose your data for 4 units of length are 13, 12, 10, and 13. Taking the entire length, we would estimate the mean number of cars per unit length $\lambda$ as $$\hat \lambda={{13+12+10+13} \over {4}}=12$$ The variance of this estimator is ${\lambda \over n},$ which we would estimate with ${\hat \lambda \over n},$ where $n$ is the number of intervals. So our estimate is $$s^2 ={ {12} \over {4}} = 3.$$

Now suppose you use intervals that are twice as long. Now your data are $25$ $(13 + 12)$ and $23$ $(10 + 13).$ The estimators are now $$\hat \lambda_1 = {{25} \over {2}}$$ and $$ \hat \lambda_2 = {{23} \over {2}} $$ If we average these we find, as before, $$\hat \lambda=12$$

Given that we have summed two estimators and divided by $2,$ the variance estimate will be ${{1} \over {4}}$ times the sum of the estimated variances. So we have $$ s^2 = {{1} \over {4}} \left({{25} \over {4}} +{{23} \over {4}}\right) =3$$

Answer 3

What you're describing here sounds like a version of the "bias variance tradeoff"

Your concern is that you are getting a free lunch by shrinking the size of your road segments by increasing the total number of segments. But remember, when you shrink the road segments you increase the variance across segments. So sure, you are increasing $N$ that, in turn, decreases the standard error in the mean but you are also increasing standard deviation since the number of cars per unit length is now quite different from segment to segment. For example, many of your segments will likely have zero cars in then as you increase $N$.

There is a related situation in online experimentation where bootstrapping a sample to compute confidence intervals is hard if the sample has tens of millions of measurements. Kleiner et al proposed a bootstrap that essentially subdivides the large sample array into $N$ non-overlapping segments, computes a metric across each segment, and takes those as bootstrap samples. So the whole operation is $O(N)$

This is equivalent to your scenario. In the Kleiner case, it's easier to see the tradeoff. The smaller $N$ is the less variance from sample to sample I will have since each segment is big enough to include open road stretches, traffic jams, etc... But I will accept a bias sample that those segment car counts will be dominated by traffic jams and I won't ever really "see" a segment of bare road.

The larger $N$ the greater variance I will experience across my bootstrap (segments with just traffic jams, some with just bare road, etc...) but I will reduce bias in that road scenarios with low car counts will surface in the samples.

The solution is to balance the two against each other to achieve your desired outcome. Do I want an accurate picture of the diversity of road scenarios or a stable measure of the general road congestion for example?

Answer 4

Ok, this is what I think is a quick and dirty answer, based on other answers and after playing around with some data.

If you have twice as many samples by making them half as long, then the mean of your count - the number of cars in each interval (not per unit length) - is halved, the SD ($s$) and SEM ($\frac{s}{\sqrt{N}}$) both decrease. But you don't want to be taking the mean count without regard of the interval size - it's a meaningless number.

If you take the mean rate - in this case, the number of cars in each interval per unit length - you get the same mean, a higher SD, but approximately the same SEM. So it works out this way.