I'm brushing up on my stats, so please bare with me (and correct me) for any mistakes. I really hope someone can help me out!
Let's consider two separate experiments that are designed to measure the length of a string.
Experiment One - (Or: How I view a statistician would determine the length of a string)
Imagine I have a population of 500 strings. I randomly sample 20 strings from this population, measure their lengths, and calculate the sample mean. I then repeat this process 100 times. By the end of the experiment, I will have 100 means, one for each time I sampled the population. This is the sampling distribution of the mean.
As I understand it, the standard deviation of this sampling distribution is the standard error of the mean. We want the standard error of the mean to be small as it means we are better zeroed in on the true population mean.
However, the standard error of the mean is also expressed as the ratio of the standard deviation of the population to the square root of the sample size (here, 20). Furthermore, it can be estimated as the ratio of the standard deviation of a single sampling of 20 strings to the square root of that sample size (again, 20).
So my question is, how does the second definition using only the standard deviations of the population or sample along with the sample size connect to the original definition in which standard error of the mean is defined as the standard deviation of our sampling distribution? I can't wrap my head around the connection.
For instance, as we conduct more and more samplings, the standard deviation of the resulting sampling distribution will continue to decrease more and more, right? So how is this fact taken into account in the equation that only uses standard deviation of a single sample divided by that sample size? Surely the standard deviation of the sampling distribution (which is the standard error!) consisting of 20000000 means will be smaller than the value we get if we simply calculate it by taking the ratio of a single sample standard deviation to the sqrt of the sample size, right?
Using the second definition, we are calculating standard error by looking at a single sample consisting of 20 measurements. But this isn't even a sampling distribution of the mean, but rather a point estimate of the mean. So how is it possible for it to even have a standard error when it's just ONE estimate?
Experiment Two - (Or: How I view a chemist/physicist would measure the length of a string)
Suppose I have a single string. I then measure that string 20 times. That's it.
Question three: In this experiment, there isn't really a 'population' from which I'm sampling. I'm just measuring the same string over and over. So how am I supposed to calculate a standard error from this? If each sampling has sample is size one, then its impossible to calculate any means nor any sampling distribution of those means. Alternatively, if we assume the 20 measurements belonged to a SINGLE sampling, then I'm still not able to construct a sampling distribution of the means, since I only got ONE mean. Sure, I could calculate the standard error of the 20 measurements, but that's not standard error, it's just the standard deviation!!
Or is it? What is it??WHAT IS ANYTHING????