Understanding standard error of the mean

It's clear to me that the formula $${SD}_\bar{x}\ = \frac{\sigma}{\sqrt{n}}$$

is equal to the true standard deviation of the sample mean, given that $\sigma$ is the population s.d., and $n$ is the sample size.

If we don't know the population standard deviation $\sigma$, best we can do is to estimate it with a sample standard deviation $s$, and the formula we get is:

$${SE}_\bar{x}\ = \frac{s}{\sqrt{n}}$$

The quantity above is called the standard error of the mean. Why error? To me this is nothing more than just an estimate of the standard deviation of the sample mean. How can this quantity be related to calculating some kind of error?

If we have the sampling distribution of the sample mean we know exactly that the mean of this distribution is equal to the population mean. Maybe the point is we don't know the population mean, then it would make sense. Because if we don't know the population mean $\mu$ AND that the unknown $\mu$ is equal to the mean of the sample mean, we could make more precise guess as of what is the population mean $\mu$ if the sample mean standard deviation had small standard deviation. Intuitively it makes sense. But why is it calculated that way? Is it possible to prove that it actually calculates the error correctly, or is it just a definition of the error?

And why making the sample size larger (increasing $n$) makes the error smaller?

General method for deriving the standard error

Difference between standard error and standard deviation

How does the standard error work?

But I still can't understand it.

• When you take a sample, how far away from the true mean would you expect its mean to be? That difference has for centuries been known as the "error" in the estimate. – whuber Apr 21 '16 at 15:44
• @whuber yeah, but I don't see how this formula answers this question. It's just standard deviation (or its estimate) divided by square root of sample size. – user5539357 Apr 21 '16 at 16:01
• @whuber let me ask a different question - is $\frac{s}{\sqrt{N}}$ the definition of the error if we talk about the mean? Or is it provable that it tells us the error? – user5539357 Apr 21 '16 at 16:38
• I gave a general description of standard error at the beginning of an answer at stats.stackexchange.com/a/18609/919 and then asserted it can be found by means of the $s/\sqrt{n}$ formula (which can be derived from that definition). Standard errors of other sample statistics have to be computed in other ways: search our site for "standard error" and peruse some of the formulas that pop up. – whuber Apr 21 '16 at 19:45
• @whuber 'which can be derived from that definition' - which definition? This one?: The standard error of the sample NPS is a measure of how much the sample NPS's typically vary between one random sample and another. – user5539357 Apr 21 '16 at 20:49