# How does the sampling distribution of sample means approximate the population mean?

I am trying to learn statistics because I find that it is so prevalent that it prohibits me from learning some things if I don't understand it properly. I am having trouble understanding this notion of a sampling distribution of the sample means. I can't understand the way some books and sites have explained it. I think I have an understanding but am unsure if its correct. Below is my attempt to understand it.

When we talk about some phenomenon taking on a normal distribution, it is generally (not always) concerning the population.

We want to use inferential statistics to predict some stuff about some population, but don't have all the data. We use random sampling and each sample of size n is equally as likely to be selected.

So we take lots of samples, lets say 100 and then the distribution of the means of those samples will be approximately normal according to the central limit theorem. The mean of the sample means will approximate the population mean.

Now what I don't understand is a lot of the times you see "A sample of 100 people…" Wouldn't we need 10s or 100s of samples of 100 people to approximate the population of the mean? Or is it the case that we can take a single sample that's large enough, say 1000 and then say that mean will approximate the population mean? OR do we take a sample of 1000 people and then take 100 random samples of 100 people in each sample from that original 1000 people we took and then use that as our approximation?

Does taking a large enough sample to approximate the mean (almost) always work? Does the population even need to be normal for this to work?

-

I think you might be confusing the expected sampling distribution of a mean (which we would calculate based on a single sample) with the (usually hypothetical) process of simulating what would happen if we did repeatedly sample from the same population multiple times.

For any given sample size (even n = 2) we would say that the sample mean (from the two people) estimates the population mean. But the estimation accuracy -- that is, how good a job we've done of estimating the population mean based on our sample data, as reflected in the standard error of the mean -- will be poorer than if we had a 20 or 200 people in our sample. This is relatively intuitive (larger samples give better estimation accuracy).

We would then use the standard error to calculate a confidence interval, which (in this case) is based around the Normal distribution (we'd probably use the t-distribution in small samples since the standard deviation of the population is often underestimated in a small sample, leading to overly optimistic standard errors.)

In answer to your last question, no we don't always need a Normally distributed population to apply these estimation methods -- the central limit theorem indicates that the sampling distribution of a mean (estimated, again, from a single sample) will tend to follow a normal distribution even when the underlying population has a non-Normal distribution. This is usually appropriate for "bigger" sample sizes.

Having said that, when you have a non-Normal population that you're sampling from, the mean might not be an appropriate summary statistic, even if the sampling distribution for that mean could be considered reliable.

-
so am I essentially just too hung up on understanding this theoretical underpinning of how some of this stuff works? Is the real interesting thing here the confidence interval? In other words if I wanted to publish a study of let's say the amount of hours the average adult in the US sleeps, and I take a sample of 5,000 and my confidence interval is 99.9% the average is between 6.46 and 6.54 then I can go ahead and publish my study saying "confidently" that the average of adult in the US sleeps 6.5 hours? –  kishinmanglani Jan 15 '13 at 23:43
Where you said: "saying "confidently" that the average of adult in the US sleeps 6.5 hours". Well, no, you can be quite confident it isn't actually 6.5 hours on average. You can only be confident it's quite close to 6.5 hours, or you could be confident that it's '6.5 hours to the nearest 5 minutes', or some such. Only ranges will have some confidence level associated with them. –  Glen_b Jan 15 '13 at 23:59
@Glen_b gets to the heart of the matter -- we can never say that we are confident that we've estimated a population value exactly right, but rather that we have some idea about the accuracy of our estimation process. –  James Stanley Jan 16 '13 at 0:27
@angrymonkey I think it is still useful to get the concepts underlying the (simulated) repeated-sampling approach. Also, for estimating means one doesn't need "huge" sample sizes -- the formula for standard error of a mean is sample std deviation / square root(n) -- the square root of n part tells us that we get diminishing returns on estimation accuracy for fixed increments as sample size gets bigger (e.g. moving from 10 to 20 people in a sample improves estimation accuracy more than going from 210 to 220 people.) –  James Stanley Jan 16 '13 at 0:31
great... thank you so much for the help. so a CI just allows us to say i am 95% sure the average person sleeps between 6.45 and 6.56 hours a night? then why do some articles make these definitive claims like the average person watches 4.5 hours of tv a day? surely the confidence interval is something like 95% 4.43 and 4.56 –  kishinmanglani Jan 16 '13 at 16:01
• If the original distribution is normal, the sample mean will also be normal, with variance $\sigma^2/n$, where $n$ is the sample size. As $n$ gets larger, the variance of the mean's distribution gets smaller, so that in the limit, the sample mean tends to the value of the population mean.
• If you take several independent samples, each sample mean will be normal, and the mean of the means will be normal, and tend to the true mean.
• If your samples are truly from the same distribution (e.g. 100 samples of 10 each), you will make the same inferences as if you took one big sample of 1000. (But in the real world, distinct samples probably do differ in ways that one cannot ignore; see "randomized block design".)
• If the data are not normal, but from a distribution that has a finite variance, then the central limit theorem implies that all the statements made above are approximately true, in the sense that the limiting distribution will be normal. The larger $n$, the closer to normality you will be.
• If you take 100 samples of 10 each, the sample means will have a distribution that is more normal looking than the original data, but less normal than the distribution of the overall mean.
• Taking a big sample will also get you close to normality.
• If you want to estimate the population mean, it makes no difference (in theory) if you take a big sample of 1000 or 100 samples of 10.
• But in practice, sampling theory people may split up the sample for reasons of clustering, stratification, and other issues. They then take the sampling scheme into account when doing their estimation. But that's really matter for another question.
-
in most textbooks, they take you through this notion of the sampling distribution of the sample means. This in essence tells you, "hey look, if you take a lot of samples, it tends to be normal, and will approximate the population mean". Then they tell you that if you take large enough samples, you can sorta only take one. Is the sampling distribution of the sample mean supposed to make you believe that you can take one large sample? In other words, what's the purpose of understanding it? Is it just to help you grasp the intuition behind taking a large sample? ignoring the idea of sampling theo –  kishinmanglani Jan 15 '13 at 22:48
I think that @"James Stanley" answers that very well. In any real life case, you take a sample, calculate the sample mean, and that's your estimate. –  Placidia Jan 15 '13 at 22:53