4
$\begingroup$

I apologise for the silliness of this question. I ran an e-learning experiment using a class of undergrads. Participation was voluntary, so only half of the class participated.

I know that my sample is the people who participated. Is the population all of the students in that class, or is the population all undergrads in similar course (or whatever I'm trying to generalise the study to)? If it's the latter, then what do I refer to the students in the class as? The cohort?

$\endgroup$
5
$\begingroup$

The population should be the collection of people that the sample was drawn from. You should only try to draw inference to the population the sample was drawn from and only if the sample is drawn at random.

$\endgroup$
  • $\begingroup$ Thank you. In my case the e-learning system was provided as a resource which supplements the course (blended learning), so participation was self-selected (besides the practicality, the justification used in the field is that this mimics what would happen in a real-world learning environment). Is it still ok to call the class the population in this case? $\endgroup$ – waitinforatrain Jul 2 '12 at 2:35
  • 2
    $\begingroup$ It would be the class but you should not try to draw inference to the population from the sample. $\endgroup$ – Michael R. Chernick Jul 2 '12 at 2:41
3
$\begingroup$

@MichaelChernick has provided a good answer here (+1). I wonder if I can provide some additional, complementary information. This will entail some subtleties. We can think of two populations: (1) the population from which the sample was drawn, and (2) the population about which you want to make a scientific statement. I will call these the 'Sample Source', and the 'Target of Inference'. Thus for you, the members of your class make up the Sample Source, and the set of all undergrads in comparable courses is the Target of Inference. I don't mean to make too big of a deal out of this, but it is easy to conflate these two, and I suspect it happens from time to time (even though your question makes clear that you didn't).

As Michael notes, the sample source is "the collection of people that the sample was drawn from". If we are going to be really strict about this, the only legitimate target population is the sample's source itself. However, such a hard line would leave us with precious little science. Major polling (such as political polling) comes about as close as we typically get to truly random samples, but they still don't really meet that ideal. We can take some solace from the fact that polling is often revealed to be pretty accurate when compared to electoral outcomes.

Given that the population you want to generalize to will pretty much always differ at least somewhat from the population from which you drew your sample, the important question is how representative your sample is of the target population. A truly random sample will be representative in every respect up to the level of sampling error. However, not every aspect of your study units will be relevant for the inferences you want to make, thus your sample need only be representative in some respects for inferences to be valid (albeit the right ones). For example, if you're wondering about the proportion of people with only one kidney (this is unusual, but within the normal range of human anatomical plasticity), a sample of college students is probably fine, but if you want to estimate mean IQ, your results will be biased because college students are not representative of humanity in that respect.

Those examples are of studies that are purely descriptive / observational, but most research is comparative / inferential (e.g., does the drug group recover faster than the placebo group). If your sample is not representative in the relevant respect, you need to think about the possible effects of those differences for the phenomenon in question. Qualitatively, there are two possibilities: First, there may be a simple shift that otherwise preserves the nature of the phenomenon. For instance, in an e-learning experiment, you may want to know whether one program promotes learning above another. With students randomized into the two programs, you can infer a relationship (e.g. $\text{program}_1>\text{program}_2$). In the first case, the higher mean IQ of college students might simply mean that all scores are better than they would be if you had more typical people, but that the relationship between the programs (e.g., the standardized mean difference) is unaffected. In this case, there is little threat to the validity of your generalizations. The second possibility is less fortuitous. It may be that the manner in which your sample differs from the target population changes the nature of the phenomenon. For instance, it may be that a particular instructional strategy works very well with high-IQ students, but makes learning more difficult for low-IQ students. Clearly, if this situation obtains, you are flat out of luck. If it makes it easier, consider the following two true data generating processes:
$$ \begin{align} \text{learning} &= \text{constant} + \beta_1\text{program}_1 + \cdots + \beta_kIQ + \varepsilon \\ \text{learning} &= \text{constant} + \beta_1\text{program}_1 + \cdots + \beta_kIQ + \beta_l\text{program}_1*IQ + \varepsilon \end{align} $$ In statistical terms, the second case has a data generating process that includes an interaction between an aspect of your sample that differs from the target population and the phenomenon under investigation, whereas the first case does not.

Note that whether your sample may differ meaningfully from the target of inference in an important respect, and if it does whether that respect interacts with the phenomenon at issue, are not primarily empirical questions. They are beliefs of the researcher. In general, there will not be tests for these assumptions in the sense that there are tests for normality or homogeneity. You simply need to be aware of these issues, and be able to make an argument for why you think your beliefs are reasonable (and realize that others might still disagree).

Hope that helps.

$\endgroup$
1
$\begingroup$

The concept of population and sample is often trickier than thought at first. I agree with Michael, the class is the population. Or, in a more general setting, a subpopulation of a larger group of individuals.

In either case it's probably incorrect to identify the group of participants as the sample, since they are not chosen at random.

I would call the class the population and the participants as a suppopulation of the class.

$\endgroup$
  • 1
    $\begingroup$ Not all samples are random - I think the participants are a sample, just not a random sample. It would be appropriate to call this a convenience sample, because it was drawn from those students available to you. $\endgroup$ – atiretoo - reinstate monica Jul 2 '12 at 17:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.