I have been occupied with a fairly simple question regarding ordinary inference procedures where my own, and many others, practice feels slightly uncomfortable. We know that the purpose of ordinary inferential methods is to deal with the situation wehre we don't have full knowledge of the population. The populaton parameters are estimated by means of a sample and some smart way to use the data from the sample. The variation and uncertainty introduced by taking a sample is also taken into account.
But we also seem to use the same methods where there is no sample, but rather some census or at least some kind of "full" sample. Let's take an example:
- The proportion of women representatives in the city council is fixed, at least at this particular moment. There is no need to calculate a confidence interval for that proportion, or to see if there is a significant difference between the proportion of men and women. These numbers are both known and fixed, they could be compared, but I can se no real need for inferential methods.
- Still, if the question was slightly more complicated, as for example whether there is any relationship between gender/age/time in the council/... and income, I would be less hesitant to jump on the inference wagon. The numbers are there, they could be put into the number crunching machine, and both coefficient estimates in a regression model and their p-values wouls turn up. And I could easily fall into interpreting these as if the background was a random sample.
How do we end up here? In neither case, there is no random sample, but still many of us would be prepared to use common inferential procedures. So my questions are fairly general:
- Is it relevant at all to use inferential statistics in cases like this?
- If so, are there any particular features in a situation where we can say that inference is relevant and valid?
I have encountered reasoning like 'this census can be regarded as a sample in time' or similar, but that usually seems farfetched and hypothetical.
Besides your general thoughts, it would be interesting to get suggestions for literature on this topic. Someone must have thought about this before.