17
$\begingroup$

I am trying to learn statistics because I find that it is so prevalent that it prohibits me from learning some things if I don't understand it properly. I am having trouble understanding this notion of a sampling distribution of the sample means. I can't understand the way some books and sites have explained it. I think I have an understanding but am unsure if its correct. Below is my attempt to understand it.

When we talk about some phenomenon taking on a normal distribution, it is generally (not always) concerning the population.

We want to use inferential statistics to predict some stuff about some population, but don't have all the data. We use random sampling and each sample of size n is equally as likely to be selected.

So we take lots of samples, lets say 100 and then the distribution of the means of those samples will be approximately normal according to the central limit theorem. The mean of the sample means will approximate the population mean.

Now what I don't understand is a lot of the times you see "A sample of 100 people…" Wouldn't we need 10s or 100s of samples of 100 people to approximate the population of the mean? Or is it the case that we can take a single sample that's large enough, say 1000 and then say that mean will approximate the population mean? OR do we take a sample of 1000 people and then take 100 random samples of 100 people in each sample from that original 1000 people we took and then use that as our approximation?

Does taking a large enough sample to approximate the mean (almost) always work? Does the population even need to be normal for this to work?

Improve this question
$\endgroup$
10
$\begingroup$

I think you might be confusing the expected sampling distribution of a mean (which we would calculate based on a single sample) with the (usually hypothetical) process of simulating what would happen if we did repeatedly sample from the same population multiple times.

For any given sample size (even n = 2) we would say that the sample mean (from the two people) estimates the population mean. But the estimation accuracy -- that is, how good a job we've done of estimating the population mean based on our sample data, as reflected in the standard error of the mean -- will be poorer than if we had a 20 or 200 people in our sample. This is relatively intuitive (larger samples give better estimation accuracy).

We would then use the standard error to calculate a confidence interval, which (in this case) is based around the Normal distribution (we'd probably use the t-distribution in small samples since the standard deviation of the population is often underestimated in a small sample, leading to overly optimistic standard errors.)

In answer to your last question, no we don't always need a Normally distributed population to apply these estimation methods -- the central limit theorem indicates that the sampling distribution of a mean (estimated, again, from a single sample) will tend to follow a normal distribution even when the underlying population has a non-Normal distribution. This is usually appropriate for "bigger" sample sizes.

Having said that, when you have a non-Normal population that you're sampling from, the mean might not be an appropriate summary statistic, even if the sampling distribution for that mean could be considered reliable.

Improve this answer
$\endgroup$
7
  • $\begingroup$ so am I essentially just too hung up on understanding this theoretical underpinning of how some of this stuff works? Is the real interesting thing here the confidence interval? In other words if I wanted to publish a study of let's say the amount of hours the average adult in the US sleeps, and I take a sample of 5,000 and my confidence interval is 99.9% the average is between 6.46 and 6.54 then I can go ahead and publish my study saying "confidently" that the average of adult in the US sleeps 6.5 hours? $\endgroup$ – mergesort Jan 15 '13 at 23:43
  • 2
    $\begingroup$ Where you said: "saying "confidently" that the average of adult in the US sleeps 6.5 hours". Well, no, you can be quite confident it isn't actually 6.5 hours on average. You can only be confident it's quite close to 6.5 hours, or you could be confident that it's '6.5 hours to the nearest 5 minutes', or some such. Only ranges will have some confidence level associated with them. $\endgroup$ – Glen_b Jan 15 '13 at 23:59
  • 1
    $\begingroup$ @Glen_b gets to the heart of the matter -- we can never say that we are confident that we've estimated a population value exactly right, but rather that we have some idea about the accuracy of our estimation process. $\endgroup$ – James Stanley Jan 16 '13 at 0:27
  • $\begingroup$ @angrymonkey I think it is still useful to get the concepts underlying the (simulated) repeated-sampling approach. Also, for estimating means one doesn't need "huge" sample sizes -- the formula for standard error of a mean is sample std deviation / square root(n) -- the square root of n part tells us that we get diminishing returns on estimation accuracy for fixed increments as sample size gets bigger (e.g. moving from 10 to 20 people in a sample improves estimation accuracy more than going from 210 to 220 people.) $\endgroup$ – James Stanley Jan 16 '13 at 0:31
  • $\begingroup$ great... thank you so much for the help. so a CI just allows us to say i am 95% sure the average person sleeps between 6.45 and 6.56 hours a night? then why do some articles make these definitive claims like the average person watches 4.5 hours of tv a day? surely the confidence interval is something like 95% 4.43 and 4.56 $\endgroup$ – mergesort Jan 16 '13 at 16:01
10
$\begingroup$
  • If the original distribution is normal, the sample mean will also be normal, with variance $\sigma^2/n$, where $n$ is the sample size. As $n$ gets larger, the variance of the mean's distribution gets smaller, so that in the limit, the sample mean tends to the value of the population mean.
  • If you take several independent samples, each sample mean will be normal, and the mean of the means will be normal, and tend to the true mean.
  • If your samples are truly from the same distribution (e.g. 100 samples of 10 each), you will make the same inferences as if you took one big sample of 1000. (But in the real world, distinct samples probably do differ in ways that one cannot ignore; see "randomized block design".)
  • If the data are not normal, but from a distribution that has a finite variance, then the central limit theorem implies that all the statements made above are approximately true, in the sense that the limiting distribution will be normal. The larger $n$, the closer to normality you will be.
  • If you take 100 samples of 10 each, the sample means will have a distribution that is more normal looking than the original data, but less normal than the distribution of the overall mean.
  • Taking a big sample will also get you close to normality.
  • If you want to estimate the population mean, it makes no difference (in theory) if you take a big sample of 1000 or 100 samples of 10.
  • But in practice, sampling theory people may split up the sample for reasons of clustering, stratification, and other issues. They then take the sampling scheme into account when doing their estimation. But that's really matter for another question.
Improve this answer
$\endgroup$
2
  • $\begingroup$ in most textbooks, they take you through this notion of the sampling distribution of the sample means. This in essence tells you, "hey look, if you take a lot of samples, it tends to be normal, and will approximate the population mean". Then they tell you that if you take large enough samples, you can sorta only take one. Is the sampling distribution of the sample mean supposed to make you believe that you can take one large sample? In other words, what's the purpose of understanding it? Is it just to help you grasp the intuition behind taking a large sample? ignoring the idea of sampling theo $\endgroup$ – mergesort Jan 15 '13 at 22:48
  • $\begingroup$ I think that @"James Stanley" answers that very well. In any real life case, you take a sample, calculate the sample mean, and that's your estimate. $\endgroup$ – Placidia Jan 15 '13 at 22:53
1
$\begingroup$

The sampling distribution of the mean is the distribution of ALL the samples of a given size. The mean of the sampling dist is equal to the mean of the population. When we talk about sampling dist of mean for samples of a given size we are Not talking about one sample or even a thousand samples but All the samples.

Improve this answer
$\endgroup$
0
$\begingroup$

The sampling dist of the mean has nothing to do with confidence intervals. That is another concept. For sampling dist the population can be normal or not normal a) If pop is normal then the samp dist of the mean will be normal for any sample size. b) If pop is Not normal then 1) the sampling dist of the mean CANNOT be considered to be normal, Unless the sample size is 30 or more. Then The Central Limit Theorem tells us the sampling dist can be considered normal.

You talk about predicting. Predicting has nothing to do with this either. You are inserting too much in samp dist. The samp dist is simply All the samples and then the mean is taken. And mean of all these samples , mu sub x bar, equals mean of population, mu and standard dev od sampling dist ,sigma sub x bar = sigma divided by square root of n. ( We won't talk about the finite pop correction factor. Take your stat for face value. Don't read too much into a concept. Fist understand the basic concept.

PS The samp dist of mean has nothing ro do abput pr

Improve this answer
$\endgroup$
1
  • $\begingroup$ I wonder if this answer could be combined w/ your 1st answer, rather than entered as another answer. We usually prefer you have 1 answer per thread. (There are exceptions, though.) You can add material to an existing answer, or make changes, by clicking the gray "edit" at its bottom left. $\endgroup$ – gung - Reinstate Monica Mar 31 '15 at 1:42
0
$\begingroup$

I have been thinking about big data problems, and looking at some of these posts this morning. I don't think this is a trivial problem at all, re the difference between analysing the 1000 data as one set compared to analysing 10 sets of 100. In theory, if the null hypothesis is true that the data are iid, it makes no difference. However, clustering and patterns in the data are not addressed at all if one simply takes the mean of the 1000 data and quotes the estimated mean and associated standard error.

The conclusion I've come to, looking at some pages on stackexchange and wikipedia, is that big data allows the obvious to be seen. If there are any interesting features in the population as a whole, a big data set would show them clear as day. So if I had a very large dataset, that I could look at visually, I wouldn't jump in and take brief summary measures without first looking for very obvious features. From my earliest lessons in statistical inference I have been taught to look at graphs and visualisations of the data as a first pass. I can't emphasise that enough. If the dataset is too big for a human to look at on a screen, then it should be sub-sampled from at a resolution that is human-readable.

Improve this answer
$\endgroup$
1
  • $\begingroup$ Please don't sign your posts -- that's what the user name at the bottom right of your post is for. $\endgroup$ – Glen_b Jun 26 '15 at 11:17

Not the answer you're looking for? Browse other questions tagged or ask your own question.