Take the 2-minute tour ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

Suppose there is a very big (infinite?) population of normally distributed values with unknown mean and variance.

Suppose also that we have a sample, S, of n values from the entire population. We can calculate mean and standard deviation for this sample (we use n-1 for stdev calculation).

The first and most important question is how is stdev(S) related to the standard deviation of the entire population?

An illustration for this issue is the second question:

Suppose we have an additional number, x, and we would like to test whether it is an vis-a-vis the general population. My intuitive approach is to calculate Z as follows:

Z = (x - mean(S))/stdev(S)

and then test it against standard distribution if n>30 or against t-distribution if n<30.

However, this approach doesn't account for n, the size of the sample. What is the right way to solve this question provided there is only single sample S?

share|improve this question
    
What do you mean by: "we would like to test whether it is an vis-a-vis the general population"? and "single sample S"? –  user28 Jul 28 '10 at 12:06

6 Answers 6

The second question seems to ask for a prediction interval for one future observation. Such an interval is readily calculated under the assumptions that (a) the future observation is from the same distribution and (b) is independent of the previous sample. When the underlying distribution is Normal, we just have to erect an interval around the difference of two Gaussian random variables. Note that the interval will be wider than suggested by a naive application of a t-test or z-test, because it has to accommodate the variance of the future value, too. This rules out all the answers I have seen posted so far, so I guess I had better quote one explicitly. Hahn & Meeker's formula for the endpoints of this prediction interval is

$$m \pm t \times \sqrt{1 + \frac{1}{n}} \times s$$

where $m$ is the sample mean, $t$ is an appropriate two-sided critical value of Student's $t$ (for $n-1$ df), $s$ is the sample standard deviation, and $n$ is the sample size. Note in particular the factor of $\sqrt{1+1/n}$ instead of $\sqrt{1/n}$. That's a big difference!

This interval is used like any other interval: the requested test simply examines whether the new value lies within the prediction interval. If so, the new value is consistent with the sample; if not, we reject the hypothesis that it was independently drawn from the same distribution as the sample. Generalizations from one future value to $k$ future values or to the mean (or max or min) of $k$ future values, etc., exist.

There is a extensive literature on prediction intervals especially in a regression context. Any decent regression textbook will have formulas. You could begin with the Wikipedia entry ;-). Hahn & Meeker's Statistical Intervals is still in print and is an accessible read.

The first question has an an answer that is so routine nobody seems yet to have given it here (although some of the links provide details). For completeness, then, I will close by remarking that when the population has approximately a Normal distribution, the sample standard deviation is distributed as the square root of a scaled chi-square variate of n-1 df whose expectation is the population variance. That means (roughly) we expect the sample sd to be close to the population sd and the difference between the two usually will be on the order of $\sqrt{n-1}$ times the population sd. Unlike parallel statements for the sample mean (which invoke the CLT), this statement relies fairly strongly on the assumption of a Normal population.

share|improve this answer

I'm finding it rather tricky to see what you are asking:

  1. If you want to know whether the Var(S) is different from the population variance, then see this previous answer.
  2. If you want to determine whether the mean(S) and the mean(X) are the same, then look at Independent two-sample t-tests.
  3. If you want to test whether mean(S) is equal to the population mean, then see @Srikant answer above, i.e. a one-sample t-test.
share|improve this answer

My first answer was full of errors. Here is a corrected version:

The correct way to test is as follows:

z = (mean(S) - mu) / (stdev(S) / sqrt(n) )

See: Student's t-test

Note the following:

  1. The sample size is accounted for when you divide the standard deviation by the square root of the sample size.

  2. You should also note that the z-test is for testing whether the true mean of the population is some particular value. It does not make sense to substitute x instead of mu in the above statistic.

share|improve this answer

I think you need to nail down the question you are asking, before you can compute an answer. I think this question is way too vague to answer: "test whether it is an vis-a-vis the general population".

The only question I think you can answer is this one: If the new value came from the same population as the others, what is the chance that it will be so far (or further) from the sample mean? That is the question that your equation will begin to answer, although it is not quite right. Here is a corrected equation that includes n.

t = (x - mean(S))/(stdev(S)/sqrt(n))

Compute the corresponding P value (with n-1 degrees of freedom) and you've answered the question.

share|improve this answer
    
I am not sure I understand what you are saying. I am not sure there is any relationship between a new draw x and the current sample mean mean(S) as x ~ N(mu,sigma^2). Clearly, the draw of x can be anywhere in the support of the distribution. It is more likely to be around mu and less likely to be in the tails but it does not have anything to do with mean(S). –  user28 Jul 29 '10 at 4:20
    
But you don't and can't know mu or sigma. What you know are the mean and SD of the sample, mean(S) and stdev(S). –  Harvey Motulsky Jul 30 '10 at 3:51

"how is stdev(S) related to the standard deviation of the entire population?"

I don't know if the "Confidence Interval" concept might be what you are looking for?

Stdev(S) is an Estimate of the standard deviation of the entire population. To see how good an estimate, confidence intervals could be computed, and these would be dependent on the sample size.

See for e.g., Simulation and the Monte Carlo Method, Rubinstein & Kroese.

share|improve this answer

1) The standard deviation of the sample (stdev(S)) is an unbiased estimate of the standard deviation of the population.

2) Given we have estimated both the population mean and variance we need to take this into account when we evaluate whether a new observation x is a member of this population. We don't use Z = (x - mean(S))/stdev(S), but rather: t = (x - mean(S))/(stdev(S)*sqrt(1 + 1/n)), where n is the sample size of the first sample. We the compare t with a t-distribution with n-1 degrees of freedom to give a p-value. See here:

http://en.wikipedia.org/wiki/Prediction_interval#Unknown_mean.2C_unknown_variance

This accounts for the sample size both in the divisor (sqrt(1 + 1/n)) and in the degrees of freedom of the t-distribution.

share|improve this answer
1  
It is the variance which is an unbiased estimator. S is not an unbiased estimate of the population standard deviation. –  Rob Hyndman Aug 18 '10 at 12:13
    
Thanks Rob, I stand corrected! This is a subtlety that I was previousy unaware of. Others may wish to see: en.wikipedia.org/wiki/Unbiased_estimation_of_standard_deviation –  Thylacoleo Aug 18 '10 at 12:41

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.