Introduction I am confused about when a sample is a random sample (i.e. probability sample) and when it is a purposive sample (i.e. non-probability sample). My understanding is that the former allows me to conduct statistical analyses and generalize to a larger population but that the latter precludes the ability to estimate a sampling error and, therefore, precludes the ability to make statistical inferences about a population.
Scenario 1 As an example, consider a regulatory agency auditing pet stores in Arizona, trying to determine whether a pet store is up-to-date on vaccinations for its animals. Thus, the inclusion criteria in the sampling frame consists of all pet stores in Arizona and the regulatory agency can randomly sample from that list and it would be a random sample, allowing them to generalize the findings to all pet stores in Arizona, correct?
Scenario 2 However, to save on costs, let's say they impose some exclusion criteria to limit the universe by removing those pet stores that are "certified" (i.e. they have passed a training program on animal vaccination), and therefore unlikely to be malfeasant. If the regulatory excludes them, the sampling frame will become "all non-certified pet stores in Arizona". If a random sample is then drawn from this limited sampling frame, is it still a random sample? I think so, with the caveat that I cannot generalize the findings to all pet stores in Arizona, but only to "all non-certified pet stores in Arizona", right?
Scenario 3 Finally, let's say that instead of exclusion criteria based on a clear-cut subgroup such as "certified" stores, what if an expert in the field curated the sampling frame, crossing off the list those stores deemed to be "unlikely offenders", based on his knowledge of the pet store industry. Once the list is curated, the remaining pet stores will be randomly selected, but this no longer seems like a random sample. Instead, it seems like it's a purposive sample, or perhaps a judgment sample, correct? If so, what are the implications for statistically analyzing the findings?