# Can someone explain to me the sampling distribution of sample variance in comparison to that of the sample mean?

I have read tons of things already about the sampling distribution of the sample variance but I can't get quite a good grasp of exactly what it is like in terms of the formulas of the measurements. Could anyone explain to me in detail what those measurements are in parallel with the sampling distribution of the sample mean?

I will be posting my understanding of the formulas of the sampling distribution of the sample mean so you guys have a reference of what information to parallel.

Sampling Distribution of the Sample Mean

Qualities

1. Distribution Type

When the population distribution is definitely or approximately normally distributed, the sampling distribution will always be normally distributed. But if the population distribution is not normally distributed, a rough guideline is followed where the sample must be equal to or greater than 30 to approximate normal distribution.

1. Standardized Test Statistic

Z Score is used when the population variance is known and the distribution is definitely or approximately normally distributed. If the distribution is not normal, the sample size must be greater than or equal to 30. T score is used if the population variance is given and distribution is not normal and the sample size is less than 30. If the population variance is not given and the sample variance is, this is also used.

Quantities

1. Mean

E(X) = μ

1. Variance

σ is given

V(X) = σ^2/n

s is given

V(X) = s^2/n

1. Standardized Test Statistic

Z Score Given σ

Z = (X̄ - μ) / (σ / √n)

Z Score Given s

Z = (X̄ - μ) / (s / √n)

T Score

t = (X̄ - μ) / (s / √n)

Here are some information I have gathered based on readings about the topic. I've been reading about the sampling distribution of the sampling variance having a chi-squared distribution with n - 1 degrees of freedom. Also, the formula of (n - 1)S^2 / σ pops up. Is this like for the chi-square distribution, where the chi-square is our standardized test statistic?

Definitely there is information online. But I have come here because I really want cohesiveness and a comparison for me to better grasp the concept.

I'll do my best to summarise some of this in a hopefully digestible way. I think some of the confusion arises from the difference between "the variance of the sample mean" and "the variance of a sample" (and potentially the variance of the variance of a sample)

1: Variance of the Sample Mean. Take a sample of size N, calculate its mean. Take another sample, calculate its mean, etc... now you have lots of sample means. The variance of the means of those samples is the variance of the sample means

2: Sample variance: Take a sample of size N. Calculate the variance within that sample

3: Variance of sample variance. As in (1), take many samples of size N, calculate all of their variances, then calculate the variance of these.

Now let's state some facts about these

Sample Mean:

You sample N times from a distribution with mean $$\mu$$ and variance $$\sigma ^{2}$$. The expected value of your sample mean is $$\mu$$, and the variance of the sample mean (see (1) above) will be $$\frac{\sigma ^{2}}{N}$$

The above holds for most underlying distributions (there are some restrictions, e.g. the mean/variance must be defined).

If the underlying distribution is Gaussian, then we can say more than just what the expected value and variance of the sample mean will be. Then we know the full distribution. The sample mean will be a normally distributed with mean $$\mu$$ and variance $$\frac{\sigma ^{2}}{N}$$ (which is consistent with what I just said), but if the distribution is not normal, then this will only be approximately true as N increases (this is the central limit theorem). The number 30 is not a good benchmark for what a good N is, it depends on the distribution.

Sample Variance:

If you take a sample of size N from a distribution and calculate the variance of the sample ((2) above), its expected value is $$\frac{N-1}{N}\sigma^{2}$$, i.e. a little bit smaller than the true distribution's variance. So if you took many samples of size N, calculated the variance within each sample and averaged these variances, you'd expect to get the above.

I'm not aware if there is a formula for the variance on the variance ((3) above).

The above holds for most distributions (as with sample mean). If however you know the underlying distribution is normal, then again, you don't just know the expected value of the sample variance, you know its full distribution, and it's given by a chi-squared distributed with (N-1) degrees of freedom, which I believe is consistent with the expected value being $$\frac{N-1}{N}\sigma ^{2}$$, although this is not as obviously trivially true as in the sample mean case above.

t-statistic (combining the two)

Now when you take a sample from a distribution, the t-statistic is a way of combining the sample mean and sample variance, $$\frac{\bar{x}}{\frac{s}{\sqrt{N-1}}}$$ where $$\bar{x}$$ is the sample mean and $$s$$ is the square root of the sample variance. This might seem like a somewhat arbitrary quantity to calculate, but it turns out that one can show that this quantity follows the t-distribution with (N-1) degrees of freedom, provided the underlying data is sample from a normal distribution.

I'm not very knowledgable about what this is used for in practice (it's called a t-test but I don't use them much, so I'll let somebody else take this part), but it involves comparing the t-statistic of different samples and then referencing them against a "t-table" to determine whether they're likely to have come from the same underlying distribution or not. This is where the number 30 comes in. For samples of size N, you must reference a "t-table with N degrees of freedom". If turns out that as N grows to about 30, a t-table starts to look very similar to a Z-table. What that means is that for N>30, the t-statistic is distributed approximately normally, i.e. like the Z-statistic. The Z-statistic involves dividing the sample mean by the distributional variance if you know it...but in practice this is never the case, when would it ever be the case that you're sampling from a distribution whose mean you don't know but whose variance you do?

Note that all of this stuff around t- and z- statistics only apply when you assume that your sample has been sampled from a normal distribution. If you don't know the underlying distribution, you can still make some assertions (subject to some assumptions about the distribution) about expected values of the sample mean, the variance of the sample mean, and expected values of the sample variance, but knowing the means and variances of distributions is less powerful than knowing the full distribution.

• Thank you for the details. Though I really wanted emphasis on the Sampling Distribution of the Sample Variance. Because I am kinda aware about the sample mean, sample variance quantities and not that confused. My intention is really to know the counterpart formulas for the Sampling Distribution of the Sample Variance. So I can have a good grasp and in a sense, make a table of formulas for Mean, Variance and Standardized Test Statistic for Sampling Distribution of Sample ___ where the blanks are Mean, Variance and Proportion. Aug 6, 2020 at 13:00
• There is just not really enough sources that make a clear cut comparison. I see sources better make comparisons with Sample Mean and Proportion but not one where the Variance is the statistic subject to sampling distribution. Nothing says, "Here are the formulas for Sampling Distribution of Sampling Variance and you can put them beside the Mean and Proportion because they work around the same concept" Aug 6, 2020 at 13:03
• Well to pull out the relevant facts: in general, you don't know anything about the sampling distributions of sample mean and variance. If however the underlying distribution is normal, then the sampling distribution of the sample mean is also normal and the sampling distribution of the sample variance is chi-squared with (N-1) degrees of freedom. The derivation of the former is pretty trivial, the latter is substantially more complex if I remember correctly. Aug 6, 2020 at 13:29

Regarding your last point, you use the formula ((n-1)S^2) / σ^2 to compute inference tests on the sample variance. And yes, it is distributed as a chi-square with n-1 DF.

• Is the chi square test statistic following the pattern of the z score and t score in terms of formula? I mean the formula of (statistic - parameter)/sd. It probably doesn't right? Especially that the chi-square is squared. I can understand the denominator being σ^2 but (n-1)S^2 doesn't look like it follows the concept of statistic - parameter. Aug 6, 2020 at 12:50
• @AndroidV11 the formula I posted is already the formula you mean. Maybe you can look here accendoreliability.com/hypothesis-tests-for-variance-case-i to this example and everything will be clear :)
– Anna
Aug 6, 2020 at 13:03
• I understand that it is. I am just asking if it follows the formula pattern of the z score and t score? Aug 6, 2020 at 13:05
• @AndroidV11 it doesn't, you you just take the formula as it is written, plug in the values and compare it with the correct chi-squared value.
– Anna
Aug 6, 2020 at 13:23

Variance is the sum of the squared errors, which is the distance from each value to the mean of the distribution. The standard deviation is the square root of the variance. A normal distribution has properties, including that the variance follows the Z table, such as 95% of the values are within 2 standard deviations (one side or the other) of the mean.

This won't always be true: "When the population distribution is definitely or approximately normally distributed, the sampling distribution will always be normally distributed." The odds of this being true increase with larger samples.

These statements aren't correct: 1) "But if the population distribution is not normally distributed, a rough guideline is followed where the sample must be equal to or greater than 30 to approximate normal distribution." 2) Z Score is used when the population variance is known and the distribution is definitely or approximately normally distributed. If the distribution is not normal, the sample size must be greater than or equal to 30."
------ The student t tables are used for small samples, and as n gets large the student t-tables becomes equivalent to the Z table (for normal populations). The student t tables have wider confidence intervals at small sample sizes, to account for the uncertainty. A population does not become normal simply because you took at least 30 samples. If you take 50 samples from an exponential distribution, it's not going to look normal (except in very improbable cases). One thing is true about sampling any type of distribution - if you take repeated samples of the same size of any population, the means of those samples will be normally distributed.

The chi-square test statistic can used for determining whether the variance of a variable in a sample is equal to the known population variance.

Re: "Is this like for the chi-square distribution, where the chi-square is our standardized test statistic?" In the same way you use a student t-table, with your degrees of freedom (n-1) and confidence level, you use the chi-square statistic with a chi-square table.

• Could you give counterpart formulas to what I have posted? The 1) Mean, 2) Variance and 3) Standardized Test Statistic Aug 6, 2020 at 12:47