How does the sampling distribution of sample means approximate the population mean?

Question

I am trying to learn statistics because I find that it is so prevalent that it prohibits me from learning some things if I don't understand it properly. I am having trouble understanding this notion of a sampling distribution of the sample means. I can't understand the way some books and sites have explained it. I think I have an understanding but am unsure if its correct. Below is my attempt to understand it.

When we talk about some phenomenon taking on a normal distribution, it is generally (not always) concerning the population.

We want to use inferential statistics to predict some stuff about some population, but don't have all the data. We use random sampling and each sample of size n is equally as likely to be selected.

So we take lots of samples, lets say 100 and then the distribution of the means of those samples will be approximately normal according to the central limit theorem. The mean of the sample means will approximate the population mean.

Now what I don't understand is a lot of the times you see "A sample of 100 people…" Wouldn't we need 10s or 100s of samples of 100 people to approximate the population of the mean? Or is it the case that we can take a single sample that's large enough, say 1000 and then say that mean will approximate the population mean? OR do we take a sample of 1000 people and then take 100 random samples of 100 people in each sample from that original 1000 people we took and then use that as our approximation?

Does taking a large enough sample to approximate the mean (almost) always work? Does the population even need to be normal for this to work?

Answer 1

I think you might be confusing the expected sampling distribution of a mean (which we would calculate based on a single sample) with the (usually hypothetical) process of simulating what would happen if we did repeatedly sample from the same population multiple times.

For any given sample size (even n = 2) we would say that the sample mean (from the two people) estimates the population mean. But the estimation accuracy -- that is, how good a job we've done of estimating the population mean based on our sample data, as reflected in the standard error of the mean -- will be poorer than if we had a 20 or 200 people in our sample. This is relatively intuitive (larger samples give better estimation accuracy).

We would then use the standard error to calculate a confidence interval, which (in this case) is based around the Normal distribution (we'd probably use the t-distribution in small samples since the standard deviation of the population is often underestimated in a small sample, leading to overly optimistic standard errors.)

In answer to your last question, no we don't always need a Normally distributed population to apply these estimation methods -- the central limit theorem indicates that the sampling distribution of a mean (estimated, again, from a single sample) will tend to follow a normal distribution even when the underlying population has a non-Normal distribution. This is usually appropriate for "bigger" sample sizes.

Having said that, when you have a non-Normal population that you're sampling from, the mean might not be an appropriate summary statistic, even if the sampling distribution for that mean could be considered reliable.

Answer 2

• If the original distribution is normal, the sample mean will also be normal, with variance $\sigma^2/n$, where $n$ is the sample size. As $n$ gets larger, the variance of the mean's distribution gets smaller, so that in the limit, the sample mean tends to the value of the population mean.
• If you take several independent samples, each sample mean will be normal, and the mean of the means will be normal, and tend to the true mean.
• If your samples are truly from the same distribution (e.g. 100 samples of 10 each), you will make the same inferences as if you took one big sample of 1000. (But in the real world, distinct samples probably do differ in ways that one cannot ignore; see "randomized block design".)
• If the data are not normal, but from a distribution that has a finite variance, then the central limit theorem implies that all the statements made above are approximately true, in the sense that the limiting distribution will be normal. The larger $n$, the closer to normality you will be.
• If you take 100 samples of 10 each, the sample means will have a distribution that is more normal looking than the original data, but less normal than the distribution of the overall mean.
• Taking a big sample will also get you close to normality.
• If you want to estimate the population mean, it makes no difference (in theory) if you take a big sample of 1000 or 100 samples of 10.
• But in practice, sampling theory people may split up the sample for reasons of clustering, stratification, and other issues. They then take the sampling scheme into account when doing their estimation. But that's really matter for another question.

Answer 3

The sampling distribution of the mean is the distribution of ALL the samples of a given size. The mean of the sampling dist is equal to the mean of the population. When we talk about sampling dist of mean for samples of a given size we are Not talking about one sample or even a thousand samples but All the samples.

Answer 4

The sampling dist of the mean has nothing to do with confidence intervals. That is another concept. For sampling dist the population can be normal or not normal a) If pop is normal then the samp dist of the mean will be normal for any sample size. b) If pop is Not normal then 1) the sampling dist of the mean CANNOT be considered to be normal, Unless the sample size is 30 or more. Then The Central Limit Theorem tells us the sampling dist can be considered normal.

You talk about predicting. Predicting has nothing to do with this either. You are inserting too much in samp dist. The samp dist is simply All the samples and then the mean is taken. And mean of all these samples , mu sub x bar, equals mean of population, mu and standard dev od sampling dist ,sigma sub x bar = sigma divided by square root of n. ( We won't talk about the finite pop correction factor. Take your stat for face value. Don't read too much into a concept. Fist understand the basic concept.

PS The samp dist of mean has nothing ro do abput pr

Answer 5

I have been thinking about big data problems, and looking at some of these posts this morning. I don't think this is a trivial problem at all, re the difference between analysing the 1000 data as one set compared to analysing 10 sets of 100. In theory, if the null hypothesis is true that the data are iid, it makes no difference. However, clustering and patterns in the data are not addressed at all if one simply takes the mean of the 1000 data and quotes the estimated mean and associated standard error.

The conclusion I've come to, looking at some pages on stackexchange and wikipedia, is that big data allows the obvious to be seen. If there are any interesting features in the population as a whole, a big data set would show them clear as day. So if I had a very large dataset, that I could look at visually, I wouldn't jump in and take brief summary measures without first looking for very obvious features. From my earliest lessons in statistical inference I have been taught to look at graphs and visualisations of the data as a first pass. I can't emphasise that enough. If the dataset is too big for a human to look at on a screen, then it should be sub-sampled from at a resolution that is human-readable.