3
$\begingroup$

I work with administrative ('routinely collected') data in situations where the data capture the entire population, e.g. hospital records, prison records, etc. When reporting statistics the question of interpreting confidence intervals has come up with colleagues and reviewers, and I'm trying to work out a coherent position. I've had a look at a related question (Why don’t we calculate the average of an entire given population instead of computing confidence interval to estimate the population mean?) but it doesn't touch on the question of measurement error.

In one sense, a confidence interval is a measure of uncertainty about our sampling process, so when the population is known, we could simply report the various statistics we use, and declare any difference as statistically significant (questions of practical significance, minimal clinical differences, etc. notwithstanding).

In another sense, there is still uncertainty in the data collection process, e.g. measures being rounded to the nearest number, which the confidence interval can stand in for. I feel this is a separate issue since it's to do with uncertainty with measuring/recording rather than sampling, but reviewers/colleagues seem to be happy to accept this interpretation of the confidence interval.

Pragmatically, the confidence intervals in these scenarios are usually incredibly small, so including them, or not, and interpreting them as including measurement uncertainty, or not, has little bearing on the conclusions we draw from data.

Overall, I'm just confused about whether confidence intervals are appropriate when dealing with known/entire/captive populations, and whether or not the concept of measurement uncertainty applies in this case or not. I don't have a strong statistics background and would appreciate any thoughts and/or references!

Improve this question
$\endgroup$
2
  • $\begingroup$ My thought: confidence intervals only apply when you've randomly sampled a portion of all the data. When you have actual measurements of the entire population you're interested in, confidence intervals wouldn't apply, but measurement intervals might. For example, if you have temperature data from thermometers known to have a normally distributed error, that would be your "confidence interval". In this case, unless you know the measurement techniques themselves are flawed, since the data is whole numbers, there are no confidence intervals. $\endgroup$
    – Barry Carter
    Commented Jul 12, 2022 at 13:23
  • $\begingroup$ @barrycarter I think this summarises my position as well, but I'm still wondering if the (sampling-related) confidence interval is a valid proxy for measurement error in this case or not... especially since there isn't really any systematic consideration given to measurement error, we just assume it's there, and that we can include it using the CIs we compute anyway. $\endgroup$
    – user2296603
    Commented Jul 12, 2022 at 13:44

1 Answer 1

Reset to default
2
$\begingroup$

A case where confidence intervals might still be useful is when the population statistic is considered as a random variable and we want to use the population statistic as a sample statistic to describe the properties of that underlying random distribution.

An example are Arbuthnot computations in births of boys and girls in London in "An argument for divine providence, taken from the constant regularity observed in the births of both sexes (1710).".

In the year 1629 the number of births in London was 5218 boys and 4683 girls. That is an exact statistic about the births in that year and not based on sampling a smaller part of the population. But those figures for the population might be interpreted as a sample from a hypothetical larger population. For example, one might wonder about the underlying distribution for the probability of a boy or a girl being born. For such a question one needs to regard the random nature of such figures, even when they relate to an entire population. This becomes also more clear when we look at multiple years

  year boys girls
  1629 5218 4683
  1630 4858 4457
  1631 4422 4102
  1632 4994 4590
  1633 5158 4839
  1634 5035 4820
  1635 5106 4928
  1636 4917 4605
  1637 4703 4457
  1638 5359 4952
  1639 5366 4784
    .    .    .
    .    .    .
    .    .    .
  1708 8239 7623
  1709 7840 7380
  1710 7640 7288
Improve this answer
$\endgroup$
12
  • $\begingroup$ So in this case, when I work with, say, the entirety of the population admitted to hospital in a particular year, they can be considered as a sample of the population admitted to hospital over time? (since the factors that lead to someone being admitted in a particular year can be considered random, I suppose) $\endgroup$
    – user2296603
    Commented Jul 12, 2022 at 13:46
  • $\begingroup$ "case where confidence intervals might still be useful is when the population statistic is considered as a random variable." This is incorrect, as by definition of CI the population statistic is not a random variable, rather the CI is a random interval. I think you are conflating credible interval with CI. $\endgroup$
    – Shreyans
    Commented May 9 at 20:49
  • $\begingroup$ @Shreyans "the population statistic is not a random variable" the response to that is in "But those figures for the population might be interpreted as a sample from a hypothetical larger population". Like in the example of the number of students that have enrolled into a school during a particular year. That number can be considered a random number and subject to many unknown influences just like the roll of a dice. $\endgroup$
    – Sextus Empiricus
    Commented May 9 at 21:33
  • $\begingroup$ @SextusEmpiricus I agree with you that the population statistic can be a random variable. I would go further and say that Bayesian credible intervals require the population statistic to be a RV (only then can we talk about prior and posterior distribution). But confidence intervals are a frequentist statistic that measure the coverage of a random interval for a fixed yet unknown population statistic. Have a look at this stats.stackexchange.com/a/6654/162449 $\endgroup$
    – Shreyans
    Commented May 9 at 22:41
  • 1
    $\begingroup$ Sidenote: confidence intervals are not defined such that the parameter to be estimated needs to be fixed. It can be a random variable (although that's a different point than what I making here in the answer). Are “Data are fixed” in Bayesian viewpoint and “Data are random” in frequentist viewpoint talking about the same thing mathematically? $\endgroup$
    – Sextus Empiricus
    Commented May 10 at 8:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.