How to estimate mean confidence intervals for a sample of a population with the population standard deviation?

Question

First off, this is my first question on stats exchange. So if I mess up or anything, please let me know because I know this website is a lifesaver, and I want to spend more time on it.

Recently, I just started working for a new company as a business analyst. Lots of our data is not available to be pulled in bulk. Pulling data for all locations in the country takes about 8 hours to complete. I hope to pull a sample from our 100+ locations and estimate the mean confidence interval. Now my understanding of this is that I would need the population standard deviation to do this (https://www.dummies.com/education/math/statistics/how-to-calculate-a-confidence-interval-for-a-population-mean-when-you-know-its-standard-deviation/). However, doesn't that mean that I would need to pull data for the entire population?

However, I read this article: Formula for confidence intervals for small samples and unknown population standard deviation. To me, it seems like what I am asking to do is feasible. However, I wondered if the formula for standard deviation would be the standard deviation of the sample? Also, would N be the size of the sample or the size of the population? Thanks for the help; my memory of statistics notation is out of date right now, haha.

Also, when I'm randomly sampling, how random should I be? Our business has locations in 8 different provinces, with a wide range of membership at each location, with a mix of urban and non-urban locations. So part of me feels that randomly selecting 10 locations (or whatever that number may be) is likely not the best route.

Thank you so much for your help. I enjoy statistics, but it has been a few years since I've done many calculations. Also, it seems a lot more difficult doing statistics for a real-life situation than a question on a test or assignment.

And once more, please let me know if I made any mistakes or missteps in this question. I want to understand how to use this site.

Thank you!!!

~ Chris

Answer 1

Suppose you have 150 locations altogether, and you decide to base your confidence interval for the mean of the population (for some attribute) from a sample of size 10.

whole = rnorm(150, 50, 7)
x = sample(whole, 10)
summary(x);  length(x);  sd(x)
   Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
  37.86   43.03   45.24   47.92   52.61   59.93 
[1] 10         # sample size
[1] 7.470816   # sample standard deviation

t.test(x)$conf.int
[1] 42.57347 53.26207
attr(,"conf.level")
[1] 0.95

The mean for the whole company is 50; a 95% confidence interval for the mean is $(42.6, 53.3).$ I used the t.test procedure in R, but the 95% CI can be found from the formula $\bar X \pm t^* S/\sqrt{n},$ where $t^* = 2.262$ cuts probability 2.5% from the upper tail of Student's t distribution with $\nu = n-1 = 9$ degrees of freedom

qt(.975, 9)
[1] 2.262157

mean(x) + qt(c(.025,.975),9)*sd(x)/sqrt(10)
[1] 42.57347 53.26207

If you knew the population standard deviation $\sigma=7,$ then you could use $\bar X \pm 1.96(7/\sqrt{10}),$ which computes to $(42.6,53.3)).$ In general, this method has the potential to be a little more accurate, but there is no difference (to one place accuracy) from the CI above for this example.

mean(x) + qnorm(c(.025,.975))*7/sqrt(10)
[1] 43.57920 52.25633

Notes: (1) You are sampling from a finite population of size 150. As long as the sample size (here $n=10)$ is less than 10% of the population size, these formulas for sampling from essentially infinite populations should give useful results.

(2) These methods assume that the population values are approximately normally distributed. These methods would not work well if you had a few locations that are hugely different from any of the others.

(3) Your idea of doing some sort of stratified sampling so several provinces are represented or that some observations are from urban and some are from rural location might be useful. That would depend on whether there are large differences among provinces or between rural or urban locations. Stratified sampling would make it somewhat more difficult to make a confidence interval.

(4) Here, because I simulated the whole population, we can find the exact population mean and standard deviation and we know that the data are normal. In most actual applications this information would not necessarily be known.

(5) If you have some data for all 100+ scores, you might try the ttest` on a sample of a dozen or so locations to how well it workd in your application.