9
$\begingroup$

This question already has an answer here:

I am conducting a descriptive study, and I am using the entire population in my study and not a sample. I understand that everything I am reporting is a population parameter at this point. However, how do I report the relationships I might find between variables. For example, if I were to set up a contingency table, how will what I find be demonstrated as being significant. Should I use chi square?

$\endgroup$

marked as duplicate by Tim, Sycorax, Silverfish, gung, whuber Nov 24 '15 at 19:11

This question has been asked before and already has an answer. If those answers do not fully address your question, please ask a new question.

  • $\begingroup$ I think that the present answers are not fairly sufficient simply because they do not provide answers on how to interpret discriptive statistics of the whole population. for example what to say about the mean and how high or low it is. $\endgroup$ – user83107 Jul 23 '15 at 17:29
15
$\begingroup$

The concept of significance or hypothesis testing is not relevant for a whole population. Hypothesis testing is based on the assumption that you deal with a sample from a (usually) infinite population, and asks the question: what is the probability that we have drawn the sample by chance from a population that fulfills the assumptions of the null hypothesis? If this probability is low, then we reject the null.

Imagine the following scenario. You measure two groups of people (for example, ten people from New York and 10 people from, say, Cracow) and find that the average height in the two groups is 1.80 and 1.79 meters, respectively, and the standard deviations are 15cm. If this is a sample from an infinite population, you will not reject the null hypothesis -- the difference is small, and we conclude that the probability of getting these results if there is no difference in reality (that is, in our infinite population) is relatively high.

However, if these two groups make the full population, then there is no significance. If you have measured every person who lives in Cracow and every person who lives in New York and you find a difference in averages of 1cm, then the populations are different in their mean, full stop. We have no probabilities any more, just measurements! (-- except possibly for a measurement error).

What you can do instead is to show the effect size. In the hypothetical example, one would show the difference between the groups for example using Cohen's d; that is, express the difference in standard deviations. In the example above, the difference would be 1cm/15cm = 0.0(6). How to calculate your effect size will depend on what actually is your data.

The point is, I think, to ask not what is statistically significant, but what effect size is significant for you as a scientist.

$\endgroup$
  • 6
    $\begingroup$ Particularly good the way you related it back to effect size and practical (rather than statistical) significance. I'd just add that there are a couple of things to watch out for: (i) when the population about which you wish to make some inference is not the population you have completely sampled (people often say 'I've sampled the population' ... but it's not the exact one they want to talk about); and (ii) when you sample a large fraction of your finite population of interest, even small biases in sampling (etc) can loom very large, dominating the (likely quite tiny) variance of an effect. $\endgroup$ – Glen_b Sep 18 '13 at 2:35
2
$\begingroup$

January's response is correct as far as it goes. However, if say the population you are considering is relatively small, say 100 to 1000 and you have collected your data for a particular period, then if you want to infer that your conclusions also apply to similar groups at a future date, then you may find it more appropriate to treat it as a sample and apply statistical processes. Even for cities, over a year, there might be a considerable influx or eflux of immigrants or emigrants, or there may be an epidemic disease or other event that could affect your conclusions, if they were used for predictive purposes.

$\endgroup$
1
$\begingroup$

You must always ask yourself, what quantities am I interested in? Statistics does not (directly) answer non-numerical questions. You must consider - which aspect of which group of people am I interested in, and how are they related to values in the sample at hand?

Descriptive statistics, such as the mean, the correlation coefficient, or Cohen's d, quantify various aspects of a sample. Inferential statistics, such as point hypothesis tests, provide estimates of these very same measures for a whole population based based on a subset (in the face of sampling error). This allows one to guess what a descriptive statistic would have been, had the whole population been measured without error. They generalise from some data we have available, to some data we haven't - under the assumption that the data we have is representative of the data we haven't.

An inferential statistic will not be more than an approximation of a descriptive statistic. As noted by @january, Statistical significance does not mean practical significance; rather, statistical significance (in the context of point hypothesis tests) informs you that you may assign low confidence to a specific single value for the population parameter (often zero). If you precisely know the value of the population parameter, I cannot imagine any reason to estimate it.

What it means to you that you have low confidence in a given value for the population parameter, such as "rejecting the null" when p<0.05, typically meaning that you reject with some confidence the hypothesis that the population mean is 0, depends on the problem in question. The value may be non-zero, but so low as to have no practical relevance. That a test rejects a point null does not carry more direct information than knowing the population parameter to be a specific (non-zero) value; rather, it carries less information, since the hypothesis test sheds doubt on one value, the descriptive statistic sheds doubt on all but one values. (Indirectly, a p value will also inform you about other descriptive statistics, such as variance and effect size). You might imagine inferential statistics as "confidence labels" assigned to descriptive statistics (although this metaphor is getting dangerously close to p(H|D)).

However, it is not so easy as to say that if the whole population the researcher is interested in has been measured, descriptive statistics are unequivocally superior. If inferential statistics make sense or not depends not only on the fraction of the population sampled, but also on the reliability of the measurements. For example, height in @January's example is rather easy to measure correctly (ignoring for now that people grow, die, have accidents, ...). But, what if you were interested in their memory span, income or beard length? In such cases, sampling error characterizes the data even though the population has been sampled, and you do not in fact precisely know the parameter. If you were to repeat the measurement, you would get completely different (though statistically very similar) results! In such cases, inference may still be useful.

But basically: consider which parameter you are interested in. p values and statistical significance aren't so much parameters, but "confidence labels" for such parameters.

$\endgroup$

Not the answer you're looking for? Browse other questions tagged or ask your own question.