Return to Answer

> #The usual unbiased estimator of the mean, 
> #for a SRS of size n = 4 is the mean.  I’ll find 
> #its exact sample distribution.
> 
> pop = c(100, 150, 50, 101, 151, 51)
> require(gtools)
> subsets = combinations(n=6, r=4)
> subsets[] = sapply(subsets, FUN=function(x){pop[x]})
> samp_dist = rowMeans(subsets)  # exact sampling distribution of sample means
> samp_dist
 [1] 100.25 112.75  87.75 125.50 100.50 113.00 100.50  75.50  88.00 100.75
[11] 113.00  88.00 100.50 113.25  88.25
> mu = mean(samp_dist)
> sigma2_sampling_dist = sum((samp_dist - mu)^2)/length(samp_dist)
>    # Note:  divided by n because this is a true variance (on a census), not an estimator
> sigma2_sampling_dist
[1] 166.6917
> 
> 
> 
> #Now consider stratification into two strata:
> 
> st1 = c(100, 150, 50)
> st2 = c(101, 151, 51)
>   # Take a SRS of size two from each stratum.  I won’t bother with
>   # combinations, as there aren’t many possible samples.  Then
>   # take the mean of each, followed by the average of these two means.
> sampling_dist1 = c(mean(100,150),mean(100,50),mean(150,50))
> sampling_dist2 = c(mean(101, 151), mean(101, 51), mean(151, 51))
> samp_dist2 = rowMeans(cbind(rep(sampling_dist1, each=3), 
+                                       rep(sampling_dist2,times=3)))
> samp_dist2
[1] 100.5 100.5 125.5 100.5 100.5 125.5 125.5 125.5 150.5
> mu2 = mean(samp_dist2)
> sigma2_sampling_dist2 = sum((samp_dist2 - mu2)^2)/length(samp_dist2)
> sigma2_sampling_dist2
[1] 277.7778

Note that the true variance of the stratification estimator is much larger than the variance of the simple random sample estimator. By the way, if I repeat this for the population $\lbrace 100, 150, 50, 150, 200, 100\rbrace$, where the strata are considerably different, I get the stratification estimator working better:

exact variance of SRS estimator: 289.1667 exact variance of stratification estimator: 277.7778

> #The usual unbiased estimator of the mean, 
> #for a SRS of size n = 4 is the mean.  I’ll find 
> #its exact sample distribution.
> 
> pop = c(100, 150, 50, 101, 151, 51)
> require(gtools)
> subsets = combinations(n=6, r=4)
> subsets[] = sapply(subsets, FUN=function(x){pop[x]})
> samp_dist = rowMeans(subsets)  # exact sampling distribution of sample means
> samp_dist
 [1] 100.25 112.75  87.75 125.50 100.50 113.00 100.50  75.50  88.00 100.75
[11] 113.00  88.00 100.50 113.25  88.25
> mu = mean(samp_dist)
> sigma2_sampling_dist = sum((samp_dist - mu)^2)/length(samp_dist)
>    # Note:  divided by n because this is a true variance (on a census), not an estimator
> sigma2_sampling_dist
[1] 166.6917
> 
> 
> 
> #Now consider stratification into two strata:
> 
> st1 = c(100, 150, 50)
> st2 = c(101, 151, 51)
>   # Take a SRS of size two from each stratum.  I won’t bother with
>   # combinations, as there aren’t many possible samples.  Then
>   # take the mean of each, followed by the average of these two means.
> sampling_dist1 = c(mean(c(100,150)),mean(c(100,50)),mean(c(150,50)))
> sampling_dist2 = c(mean(c(101, 151)), mean(c(151, 51)), mean(c(101,51)))
> samp_dist2 = rowMeans(cbind(rep(sampling_dist1, each=3), 
+                                       rep(sampling_dist2,times=3)))
> samp_dist2
[1] 125.5 113.0 100.5 100.5  88.0  75.5 113.0 100.5  88.0
> mu2 = mean(samp_dist2)
> sigma2_sampling_dist2 = sum((samp_dist2 - mu2)^2)/length(samp_dist2)
> sigma2_sampling_dist2
[1] 208.3333

Note that the true variance of the stratification estimator is much larger than the variance of the simple random sample estimator. By the way, if I repeat this for the population $\lbrace 100, 150, 50, 170, 220, 120\rbrace$, where the strata are considerably different, I get the stratification estimator working better:

exact variance of SRS estimator: 289.1667

exact variance of stratification estimator: 208.3333

The other answer is wrong, unfortunately. Actually, that should be obvious, otherwise no statistician would ever bother to do SRS and they would happily stratify on utterly useless variables- why not? (How about this- for each sampling unit in your frame, toss a coin. If you get heads, put it in stratum 1, if tails put it in stratum 2. Does it seem like doing that should increase your accuracy?)

In practice, I tell researchers that as long as they have a reasonable sample size available from each strata, and the stratification variable makes sense (they are sure that means of one stratum are 'significantly different' from those of other strata), then stratify.

Side note: While lowering variances is the usual reason to stratify, there are others. First, if you want to guarantee sufficient sample size in each stratum so that you can make separate inferences on each one, you should stratify. Second, if costs vary greatly from one stratum to another, you can stratify to optimize costs. Finally, if variability is known to be much higher in some strata than others, you can use stratification (by increasing sample size in the most variable groups) to lower your se. However, if costs and variables don't distinguish your strata, you can definitely get a wider confidence interval if you stratify on an unhelpful variable.

The other answer is wrong, unfortunately. Actually, that should be obvious, otherwise no statistician would ever bother to do SRS and they would happily stratify on utterly useless variables- why not?

In practice, I tell researchers that as long as they have a reasonable sample size available from each strata, and the stratification variable makes sense (they are sure that means of one strata are 'significantly different' from those of other strata), then stratify.

Side note: While lowering variances is the usual reason to stratify, there are others. First, if you want to guarantee sufficient sample size in each stratum so that you can make separate inferences on each one, you should stratify. Second, if costs vary greatly from one stratum to another, you can stratify to optimize costs. Finally, if variability is known to be much higher in some strata than others, you can use stratification (by increasing sample size in the most variable groups) to lower your se. However, if costs and variables don't distinguish your strata, you can definitely get a wider confidence interval if you stratify on an unhelpful variable.

I'll illustrate with an exact computation: I'll start with the population $\lbrace 100, 150, 50, 101, 151, 51\rbrace$. First I'll enumerate the EXACT sampling distribution of the means of all possible Simple Random Samples (SRS) of size n = 4 from the population. Then I'll break this into two strata, each of size three. I'll enumerate all possible means of samples based on an SRS of size 2 from each stratum.

Finally, I'll compute the exact 'population' variance (i.e. sigma squared) of each statistic.

> #The usual unbiased estimator of the mean, 
> #for a SRS of size n = 4 is the mean.  I’ll find 
> #its exact sample distribution.
> 
> pop = c(100, 150, 50, 101, 151, 51)
> require(gtools)
> subsets = combinations(n=6, r=4)
> subsets[] = sapply(subsets, FUN=function(x){pop[x]})
> samp_dist = rowMeans(subsets)  # exact sampling distribution of sample means
> samp_dist
 [1] 100.25 112.75  87.75 125.50 100.50 113.00 100.50  75.50  88.00 100.75
[11] 113.00  88.00 100.50 113.25  88.25
> mu = mean(samp_dist)
> sigma2_sampling_dist = sum((samp_dist - mu)^2)/length(samp_dist)
>    # Note:  divided by n because this is a true variance (on a census), not an estimator
> sigma2_sampling_dist
[1] 166.6917
> 
> 
> 
> #Now consider stratification into two strata:
> 
> st1 = c(100, 150, 50)
> st2 = c(101, 151, 51)
>   # Take a SRS of size two from each stratum.  I won’t bother with
>   # combinations, as there aren’t many possible samples.  Then
>   # take the mean of each, followed by the average of these two means.
> sampling_dist1 = c(mean(100,150),mean(100,50),mean(150,50))
> sampling_dist2 = c(mean(101, 151), mean(101, 51), mean(151, 51))
> samp_dist2 = rowMeans(cbind(rep(sampling_dist1, each=3), 
+                                       rep(sampling_dist2,times=3)))
> samp_dist2
[1] 100.5 100.5 125.5 100.5 100.5 125.5 125.5 125.5 150.5
> mu2 = mean(samp_dist2)
> sigma2_sampling_dist2 = sum((samp_dist2 - mu2)^2)/length(samp_dist2)
> sigma2_sampling_dist2
[1] 277.7778

Note that the true variance of the stratification estimator is much larger than the variance of the simple random sample estimator. By the way, if I repeat this for the population $\lbrace 100, 150, 50, 150, 200, 100\rbrace$, where the strata are considerably different, I get the stratification estimator working better:

exact variance of SRS estimator: 289.1667 exact variance of stratification estimator: 277.7778

Actually, it probably would have been easier to just prove this than give an example. But this shows that stratification can fail to give a lower variance estimator. Note that this example is extreme in that the sample sizes are small.

The other answer is wrong, unfortunately. Actually, that should be obvious, otherwise no statistician would ever bother to do SRS and they would happily stratify on utterly useless variables- why not?

It is true that the sum of squared error inside each stratum will drop (i.e. $\sum (x_i - {\overline{x}})^2$ will drop virtually always). This is similar to the fact that adding new variables to a regression will just about always lower the R-squared even if the variable is complete noise. However, the value of $s^2$ also involves dividing by $n_1 - 1$ and not by $n_i$. In other words, if you have $k$ strata, you lose $k-1$ degrees of freedom. So your estimated variances might rise if the stratification variable is sufficiently useless. (The effect is most serious if sample sizes are small in some strata. In the extreme, if they drop to $n_i = 1$, you can't even estimated the standard error.)

Crudely, a fast way to tell if stratification would help is to run an ANOVA on the stratification variable. If it's significant (or, at least, the ADJUSTED R-squared rises), the stratification might help.

In practice, I tell researchers that as long as they have a reasonable sample size available from each strata, and the stratification variable makes sense (they are sure that means of one strata are 'significantly different' from those of other strata), then stratify.

Side note: While lowering variances is the usual reason to stratify, there are others. First, if you want to guarantee sufficient sample size in each stratum so that you can make separate inferences on each one, you should stratify. Second, if costs vary greatly from one stratum to another, you can stratify to optimize costs. Finally, if variability is known to be much higher in some strata than others, you can use stratification (by increasing sample size in the most variable groups) to lower your se. However, if costs and variables don't distinguish your strata, you can definitely get a wider confidence interval if you stratify on an unhelpful variable.

The other answer is wrong, unfortunately. Actually, that should be obvious, otherwise no statistician would ever bother to do SRS and they would happily stratify on utterly useless variables- why not? (How about this- for each sampling unit in your frame, toss a coin. If you get heads, put it in stratum 1, if tails put it in stratum 2. Does it seem like doing that should increase your accuracy?)

It is true that the sum of squared error inside each stratum will drop (i.e. $\sum (x_i - {\overline{x}})^2$ will drop virtually always). This is similar to the fact that adding new variables to a regression will just about always lower the R-squared even if the variable is complete noise. However, the value of $s^2$ also involves dividing by $n_1 - 1$ and not by $n_i$. In other words, if you have $k$ strata, you lose $k-1$ degrees of freedom. So your estimated variances might rise if the stratification variable is sufficiently useless. (The effect is most serious if sample sizes are small in some strata. In the extreme, if they drop to $n_i = 1$, you can't even estimated the standard error.)

Crudely, a fast way to tell if stratification would help is to run an ANOVA on the stratification variable. If it's significant (or, at least, the ADJUSTED R-squared rises), the stratification might help.

In practice, I tell researchers that as long as they have a reasonable sample size available from each strata, and the stratification variable makes sense (they are sure that means of one stratum are 'significantly different' from those of other strata), then stratify.

Side note: While lowering variances is the usual reason to stratify, there are others. First, if you want to guarantee sufficient sample size in each stratum so that you can make separate inferences on each one, you should stratify. Second, if costs vary greatly from one stratum to another, you can stratify to optimize costs. Finally, if variability is known to be much higher in some strata than others, you can use stratification (by increasing sample size in the most variable groups) to lower your se. However, if costs and variables don't distinguish your strata, you can definitely get a wider confidence interval if you stratify on an unhelpful variable.