# Why take multiple samples from a population?

I'm fairly new to Econometrics, and I'm curious about, when trying to estimate the population parameter, why it is important to take multiple samples of the population, instead of just combining all of those samples into one large sample? E.g. If you were trying to estimate the effect of one extra year of education on wages, and your population is college students, why take 10 samples of 1000 students each instead of just taking 1 sample of 10,000 students? Thanks. (Sorry if this has been answered already, I couldn't find similar questions after doing a search.)

• I think you are missing some context here. No one really takes 10 samples of 1000 students unless there is some sort of cluster sampling going on here or unless someone is trying to obtain bootstrap estimates for variance estimation. – StatsStudent Mar 8 '15 at 23:36
• 10 and 1000 were just random numbers I picked to construct an example. I suppose that actually the numbers would be much higher. – randomUser47534 Mar 8 '15 at 23:41
• OK. Can you provide a bit more background on this? It's still not clear from your question. It would make sense, for e.g., to take "samples of 10 of 1000 students if you wanted to perform cluster sampling." Cluster samples are created typically when you don't have a complete listing of all the students. For example, you may not have a complete list of all college students in the US, but you might have a list of all colleges. Then it makes sense to sample first 10 colleges and then within each college, sample 1000 students because one a school is selected you obtain a list of students. – StatsStudent Mar 8 '15 at 23:43
• There will be (subtle) differences between $k$ samples, each of size $m$, and one sample of size $mk$, so various interesting and potentially useful things could be said here. But could you explain the background a little? For instance, where exactly is this procedure of taking a bunch of small samples recommended? For what purpose(s)? – whuber Mar 8 '15 at 23:45
• I'm afraid we won't be able to help unless you provide a bit more background and clarification. Could you explain a bit more about where you may have seen this done? We understand you were using colleges and students as an example. That's not the point of confusion here. The point is we have no context with your question to help you out and there are multiple "answers" we could give depending on where you are headed here. – StatsStudent Mar 8 '15 at 23:47

Let's present a case where it is preferable to use the pooled-data estimator. This is related to an old question of mine so some things are repeated.

Pooling the available data of total size $n$ against keeping it separate and work with $m$ smaller samples each of size $n_j = n/m$ has an advantage in the context of OLS regression, as regards the variance of the OLS estimator, at least when the $m$ samples are considered independent.

Consider a linear regression setting (to which the OP alludes), and let's say that all nice properties of the OLS estimator do hold: unbiasedness, efficiency, consistency.

Let's say we have two options: take one sample of size $n$ or $m$ samples each of size $n_j = n/m$. If we take one sample, we will run one regression,

$$\mathbf y = \mathbf X\beta + \mathbf u,\;\; {\rm Var}(\mathbf u\mid X) =\sigma^2\mathbf I$$

and we will obtain a single estimate, $\hat \beta$.

If we take $m$ independent samples we will run $m$ independent regressions,

$$\mathbf y_j = \mathbf X_j\beta + \mathbf u_j, \;\;{\rm Var}(\mathbf u_j \mid X_j) =\sigma^2\mathbf I,\;\; j=1,...m$$

and we will get $m$ estimates $b_j, j=1,...,m$. This looks better, we can obtain an empirical distribution, rather than just a single estimate. But what if we want eventually to obtain, from this route also, a single estimate? It appears that the most natural thing would be to average over the lot, arriving thus at the averaging estimator

$$\bar b = \frac 1m\sum_{j=1}^{m}b_j$$

All estimators involved in both approaches are unbiased and consistent. But informally, in the first case, we appeal to the consistency property of the estimator: by using a large sample we hope that asymptotic properties will affect beneficially the accuracy of the estimate that we will obtain.

In the second case, we appeal to the unbiasedness property of the estimators: obtain many estimates and take the average, estimating the expected value of the estimator which equals the true value.

What happens is that
$${\rm Var}(\hat \beta \mid \mathbf X) < {\rm Var}(\bar b \mid \mathbf X)$$

in the matrix sense, i.e. that the difference ${\rm Var}(\bar b \mid \mathbf X) -{\rm Var}(\hat \beta \mid \mathbf X)$ is a positive definite matrix: The estimator from the single sample has lower variance than the averaging estimator.

Consider first

$${\rm Var}(\bar b \mid \mathbf X) = {\rm Var}\left(\frac 1m\sum_{j=1}^{m}b_j\mid \mathbf X\right) = \frac 1{m^2}\sum_{j=1}^{m}{\rm Var}(b_j\mid \mathbf X) = \sigma^2\frac 1{m^2}\sum_{j=1}^{m}(\mathbf X_j' \mathbf X_j)^{-1}$$

No covariances appear because the samples are assumed independent. Using the symbol $A$ to denote the arithmetic mean, we can write

$${\rm Var}(\bar b\mid \mathbf X) = \frac {\sigma^2}{m}\cdot A\left[\left(\mathbf X_j' \mathbf X_j\right)^{-1};j=1,...,m\right] \tag{1}$$

For the single-sample estimator we have

$${\rm Var}(\hat \beta\mid \mathbf X) = \sigma^2 (\mathbf X' \mathbf X)^{-1}$$

Now, $\mathbf X$ is a matrix that stacks the $\mathbf X_j$ matrices of the $m$ samples,

$$\mathbf X = \left [ \begin{matrix} \mathbf X_1 \\ \mathbf X_2\\ . \\ . \\ \mathbf X_m \end{matrix}\right]$$

Therefore,

$$\mathbf X' \mathbf X = \mathbf X_1' \mathbf X_1 + \mathbf X_2' \mathbf X_2 +...+\mathbf X_m' \mathbf X_m$$

Manipulate this into

$$\mathbf X' \mathbf X = \left[\left(\mathbf X_1' \mathbf X_1\right)^{-1}\right]^{-1} + \left[\left(\mathbf X_2' \mathbf X_2\right)^{-1}\right]^{-1} +...+\left[\left(\mathbf X_m' \mathbf X_m\right)^{-1}\right]^{-1}$$

$$\implies \left(\mathbf X' \mathbf X\right)^{-1} = \left(\left[\left(\mathbf X_1' \mathbf X_1\right)^{-1}\right]^{-1} + \left[\left(\mathbf X_2' \mathbf X_2\right)^{-1}\right]^{-1} +...+\left[\left(\mathbf X_m' \mathbf X_m\right)^{-1}\right]^{-1}\right)^{-1}$$

After our eyes adjust to the three layers of inverses, we can see that the right hand side is the scaled harmonic mean of the $\left(\mathbf X_j' \mathbf X_j\right)^{-1}$ matrices. Note that this is the harmonic mean in the matrix sense of the term.

Using the symbol $H$ to denote the harmonic mean we have

$${\rm Var}(\hat \beta\mid \mathbf X) = \sigma^2 (\mathbf X' \mathbf X)^{-1} = \frac {\sigma^2}{m}\cdot H\left[\left(\mathbf X_j' \mathbf X_j\right)^{-1};j=1,...,m\right] \tag{2}$$

For scalars, it is well known that $H<A$ always. It has been proven that the same holds for matrices too. So we have proven that

$${\rm Var}(\hat \beta\mid \mathbf X) < {\rm Var}(\bar b \mid \mathbf X)$$ again in the matrix sense.

So in this set up, we prefer to pool the data because it lowers the variance.

Note: "pooling the data" does not always produce this beneficial effect. For example, if we want to estimate the mean of a population, the variance of the sample mean from a single-sample of size $n$ is the same as the variance of the average of $m$ sample means from $m$ samples each of size $n/m$.

• Nice answer (+1). When you say "pooling data", do you imply a resampling procedure, such as bootstrapping or jackknife, or some other one of similar nature? – Aleksandr Blekh Mar 9 '15 at 13:56
• @AleksandrBlekh No, just putting all the collected data together in a single sample, instead of examining separate samples. – Alecos Papadopoulos Mar 9 '15 at 13:58
• Great answer, but based on the OP question, I'm guessing this is way over his or her head. – StatsStudent Mar 10 '15 at 3:15

I think this is a common confusion among new students of statistics that OP is experiencing. At least I had a hard time wrapping my head around it. The difficult concept is that of the sampling distribution of an estimator.

In order to explain sampling distributions of regression parameters teachers often explain by saying "if we drew a large number of samples of size n from the population of interest and performed regressions on the individual samples then our estimates obtained from our regressions would, in expectation (on average), equal the true population parameter. By using the central limit theorem we can construct confidence intervals for the parameters. Etc."

This is not what people do in actual studies, it is just a thought experiment to illustrate that our estimates of $\beta$, or whatever other parameter we are looking for, come from a sampling distribution and that these estimates will vary, in a certain way described nicely by the central limit theorem, around the true parameter we are looking for.

In actual studies data is almost always "pooled", meaning we use all the observations we have at the same time.

There are situations where it can be much better to take multiple samples of a population rather than a single large one: in agencies where the smaller samples are spread out over time. I've known of organizations that take one large client survey during a year, usually in a single month. In one agency the number was 1,800 out of 9,000 clients, who were given survey forms when they came in for monthly benefits and counseling. There was no time for staff to answer questions about the survey; they were too busy doing their regular jobs; response was low; and the quality of the data were poor. The final numbers, ~1,000, always looked impressive large, but the bias was never investigated.

In contrast, another organization took monthly smaller surveys. The number of clients sampled was never more than 50 a month. They were picked at random in advance of their appointments and asked to come in 10 minutes early. A single staffer could administer the interview. She and one other devoted part of their time to managing the survey and to personalized follow ups. contacts. This process had two main benefits from the organization's point of view: 1) Though the monthly samples were small, response was high, and the overall quality of the data was higher; 2) Perhaps more important, the survey information was *current(; it was easy to keep keep running statistics for the survey month, the past two months, and even longer periods. The agency higher ups, knowing the quality of the survey, actually paid attention to the results and suggested topics.