In stats classes, I've always learned that a population is always a very broad, almost unquantifiable group (e.g., all voters in a country, all consumers of a company, all viewers of a TV channel), which is why we use samples to estimate population trends.

But in examples where we are able to have all the current information, but there will be more information in the future, do we treat the information we have as a population or sample?

For example, say you're analyzing the results of a game show and trying to estimate how women perform on it vs. men. Say the game show has only been played ~50 times, and we have all the data from it, but more games will be played in the future. Would you be able to treat those 50 games already played as a sample and run statistical tests on them, even though it's technically all of the information for that game that exists? Do the individuals of a population have to be ones that "exist" in the real time, or can you interpret the population in this case as all the iterations of the game show that are "hanging out in the ether" that just haven't been played yet?

Also, if we did treat the first 50 games as a sample, would that violate the sample's ability to be considered "random"?

  • 1
    $\begingroup$ Welcome to this community! I recommend this article by Lindley & Novick: projecteuclid.org/euclid.aos/1176345331 They have an interesting and useful take on the notion of (super)population. $\endgroup$ – pglpm Jun 13 at 13:53

It is true that populations are usually broad but they do not have to be. For example, according to Newbold, Carlson, & Thorne (2013) Statistics for business and economics, textbook:

A population is the complete set of all items that interest an investigator. Population size, $N$, can be very large or even infinite. A sample is an observed subset (or portion) of a population with sample size given by $n$.

The first sentence is the most important. What is population depends on what is of your interest. If a person's interest is to examine how their own children change their behavior when introduced to some reward, then the population would be the number of their own children even if the number of children might be small 1-2 for example. However, in statistics and econometrics more general we are most of the time interested in answering broad useful questions. In the above example it is not very interesting from scientific perspective to discover how some reward changes behavior of your own children, a scientist would most likely want to know how it changes behavior of all children.

However, in your case since you are interested in broad terms how women and men perform on that particular show then your population actually would consist of all possible contestants and you can treat those 50 observations as a sample. Whether we can call this sample random (in sense of simple random sapling) depends on whether you can justify that in principle any member of a population had equal chance of being selected for the show.

| cite | improve this answer | |
  • 2
    $\begingroup$ equal chance of being selected is not a requisite for random sampling. $\endgroup$ – carlo Jun 12 at 15:31
  • $\begingroup$ @carlo yes you are right but it is required for simple random sampling and often when people talk about simple random sapling they just say random sampling - I wasn’t sure if this is what OP meant - that’s why in my answer I qualified that in the brackets that I am talking in sense of simple random sample $\endgroup$ – 1muflon1 Jun 12 at 15:34

The gist is that the instances of the game show are a sample of the results that conceivable could have happened.

Think about flipping a coin and trying to figure out the probability of getting H or T. You’ve flipped 50 times and gotten 30 H and 20 T. It is inarguable that you got H on 60% of the flips. However, it is conceivable that the coin’s true probability of H, for the population, is 61% or 59%.

This is why you might use inferential statistics even though you have observed every instance. You want to know about the data-generating process (DGP), not just the data that happened to be generated.

| cite | improve this answer | |
  • $\begingroup$ In fact, it's conceivable that P(heads) is just about any number between 0 and 1, exclusive, though the likelihood of it being such a number diminishes as the number drifts away from 60%. $\endgroup$ – Nij Jun 12 at 23:27
  • $\begingroup$ @Nij I am not gambling with you if you throw heads with probability 60%. $\endgroup$ – Nick Cox Jun 13 at 8:43
  • $\begingroup$ Why not? I'd take 2:1 if you were flipping heads at 60%. $\endgroup$ – Nij Jun 13 at 9:51

Others have answered perfectly to the first question, I'll consider the part about random samples:

No, your sample is not random.

But don't worry, it's fine.

Statisticians use non-random samples all the time! For instance, typically, when you do research about some disease progression, you decide a date and you pick every patient you have in one specific hospital, from that date on, until you get a sample as large as you want. That's the sample and it's not random at all! It doesn't consider patients from other time spans and from other hospitals, so what's the point on making inference on it?

The point is to get an idea of what is the data generating process: you know that any patient could have had complications, so you want to discover what's the chances for each of them, what are the risk factors, etc... Your results stand true, in principle, just for people who get treated in that hospital in that period (not a useful population, is it?), but truth is, the usefulness of the study depends on the degree of belief you have that what's good for those patients is also good for other people who got sick around the world.

Coming back to your case, if you believe that competitors in that show will keep behaving similarly, than you can trust on analysis carried on that data. On the other hand you can expect competitors to learn from the past and change their behaviour. It's really hard to tell, but in that case your analysis may be less reliable.

That's also true for medical research, by the way. Patients around the world are not granted to react to illness and treatment the same way a sample from Swedland does.

| cite | improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.