1
$\begingroup$

I work with administrative ('routinely collected') data in situations where the data capture the entire population, e.g. hospital records, prison records, etc. When reporting statistics the question of interpreting confidence intervals has come up with colleagues and reviewers, and I'm trying to work out a coherent position. I've had a look at a related question (Why don’t we calculate the average of an entire given population instead of computing confidence interval to estimate the population mean?) but it doesn't touch on the question of measurement error.

In one sense, a confidence interval is a measure of uncertainty about our sampling process, so when the population is known, we could simply report the various statistics we use, and declare any difference as statistically significant (questions of practical significance, minimal clinical differences, etc. notwithstanding).

In another sense, there is still uncertainty in the data collection process, e.g. measures being rounded to the nearest number, which the confidence interval can stand in for. I feel this is a separate issue since it's to do with uncertainty with measuring/recording rather than sampling, but reviewers/colleagues seem to be happy to accept this interpretation of the confidence interval.

Pragmatically, the confidence intervals in these scenarios are usually incredibly small, so including them, or not, and interpreting them as including measurement uncertainty, or not, has little bearing on the conclusions we draw from data.

Overall, I'm just confused about whether confidence intervals are appropriate when dealing with known/entire/captive populations, and whether or not the concept of measurement uncertainty applies in this case or not. I don't have a strong statistics background and would appreciate any thoughts and/or references!

$\endgroup$
2
  • $\begingroup$ My thought: confidence intervals only apply when you've randomly sampled a portion of all the data. When you have actual measurements of the entire population you're interested in, confidence intervals wouldn't apply, but measurement intervals might. For example, if you have temperature data from thermometers known to have a normally distributed error, that would be your "confidence interval". In this case, unless you know the measurement techniques themselves are flawed, since the data is whole numbers, there are no confidence intervals. $\endgroup$ Jul 12, 2022 at 13:23
  • $\begingroup$ @barrycarter I think this summarises my position as well, but I'm still wondering if the (sampling-related) confidence interval is a valid proxy for measurement error in this case or not... especially since there isn't really any systematic consideration given to measurement error, we just assume it's there, and that we can include it using the CIs we compute anyway. $\endgroup$ Jul 12, 2022 at 13:44

1 Answer 1

1
$\begingroup$

A case where confidence intervals might still be useful is when the population statistic is considered as a random variable.

For instance the number of students that enrolled into a school. You can count them precisely, or at least without much error and with no errors due to random sampling. But those figures for the population might be interpreted as a sample from a hypothetical larger population. For instance the school board might wonder whether the numbers of students are following an increasing trend. For such a question one needs to regard the random nature of such figures, even when they relate to an entire population.

$\endgroup$
1
  • $\begingroup$ So in this case, when I work with, say, the entirety of the population admitted to hospital in a particular year, they can be considered as a sample of the population admitted to hospital over time? (since the factors that lead to someone being admitted in a particular year can be considered random, I suppose) $\endgroup$ Jul 12, 2022 at 13:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.