Trying to understand the intuition behind the standard error of the mean.

Starting from this formula:

$Var(\bar{X}) = Var(\frac{\bar{X}_1+\bar{X}_2+...+\bar{X}_n}{n})$

The formula talks about a variance, specifically of observations divided by the number of observations (which is their mean).

The fact is that the mean of a single sample is a single value.

Shouldn't his variance ($Var(\bar{X})$), be a single value, therefore equal to 0?

  • 1
    $\begingroup$ When you repeat the process of obtaining a sample of one observation, the results usually change: thus the variance cannot possibly be zero! $\endgroup$ – whuber Nov 10 '20 at 14:22
  • 1
    $\begingroup$ Note that you are calculating $Var(\bar{X})$ not $Var(\bar{x})$. $\bar{X}$ denotes the random variable from repeated samples, $\bar{x}$ is a realized instance of this random variable. $\endgroup$ – Dayne Nov 10 '20 at 16:35
  • $\begingroup$ Hey whuber, Dayne thank you for your reply. So if I understand correctly Dayne, X¯ stands for multiple samples, than the second part of the formula is the different means of multiple samples, divided by the numerosity of the samples; and not a simple mean of the multiple instances of one sample like a thought (and like whuber thought if I'm correct). If that is the case, what's the meaning of dividing by the numerosity, given that is not a mean? $\endgroup$ – Alberto Lugli Nov 12 '20 at 9:42
  • $\begingroup$ It is like taking all the means of the different samples, make them into a single one, dividing this amount equally for the single instances of the samples, and trying to obtain a variance from this single number, this single odd* amount of "mean of the means distributed equally for the number of instances of only one sample". *Odd because it's not clear in my mind. $\endgroup$ – Alberto Lugli Nov 12 '20 at 9:44
  • $\begingroup$ @Dayne, tagging you properly so that you receive a notification $\endgroup$ – Alberto Lugli Nov 15 '20 at 8:24

This is my intuition behind it. The standard error of the mean is

$$ SEM = \frac{s}{\sqrt{n}} $$

with $n$ being the sample size and $s$ the standard deviation of the sample.

The sample you have is one of many possible samples you could have collected from the entire population (which is typically unavailable or unknown). The mean of this sample will differ from the mean of the population due to (at least) the chance involved in collecting it. Therefore, you may want to know how much discrepancy you can expect between the mean of the sample and the true population mean and this is what the SEM is telling you. Intuitively, the larger the sample the more accurate it will be in estimating the population mean so $n$ is at the denominator. In contrast, if the variation in the sample ($s$) is large we should expect a more noisy estimate so $s$ is at the numerator.

Here's an example in R which should be understandable even if you don't know R:

pop <- rnorm(n= 10000, mean= 0, sd= 1) # Entire population 
pop[1:10] # First 10 values
# -1.21  0.28  1.08 -2.35  0.43  0.51 -0.57 -0.55 -0.56 -0.89 

Collect 50 samples, each of size 50, and calculate mean and SEM for each:

N <- 50
means <- rep(NA, N)
stderr <- rep(NA, N)
for(i in 1:N) {
    smp <- sample(pop, size= 50)
    means[i] <- mean(smp)
    stderr[i] <- sd(smp) / sqrt(length(smp))

Plot them together with true population mean (blue line, unknown in real life):

plot(x= 1:N, y= means, pch= 19, ylim= c(-0.5, 0.5))
segments(x0= 1:N, x1= 1:N, y0= means-stderr, y1= means+stderr)
abline(h= mean(pop), lty= 'dashed', col= 'blue')

enter image description here

As you would expect, samples are different from each other and from the true population mean

If the original population has little variation, you can get a more accurate estimate of the mean for the same sample size. This is the same plot but using pop <- rnorm(n= 10000, mean= 0, sd= 0.5) (sd reduced from 1 to 0.5):

enter image description here

  • $\begingroup$ Hey @dariober, thank you very much for the time you took for the reply. I can grasp the two concepts you write here. Still I have some issues because it's not clear to me what's the meaning of that n. It's not a mean because n should be "number of samples" instead of "numerosity of sample". And how is possible to make a varinace of a single number. As an example when I talk about intuition of the concept of the variance my brains visualize something along the lines of this post of Fahd Alhazmi:towardsdatascience.com/… $\endgroup$ – Alberto Lugli Nov 16 '20 at 15:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.