# Why cannot a sample of voluntary patients be generalized?

In the book “OpenIntro Statistics” it is specified that samples of voluntary patients cannot be generalized.

Why cannot a sample of voluntary patients be generalized?

-
It might be illuminating to turn your question around: could you explain the reasoning that would support "generalization" from a sample of voluntary patients? To what population would you generalize and why would your generalization be valid? – whuber Jul 3 '14 at 16:37
@whuber, Done! Thanks. – SunnyShah Jul 3 '14 at 16:40
I'm sorry, I wasn't meaning to suggest that you ask us that inverted question: please ask yourself that question. In thinking about it you should be able to come up with a good answer to your original question. – whuber Jul 3 '14 at 16:43
Ok! Thanks. Understood. I would still like to have an answer from an expert about it. – SunnyShah Jul 3 '14 at 16:46
It's a matter of selection bias. – Sergio Jul 3 '14 at 17:29

With no additional information, I am left to assume that the author is referring to the difficulty in performing statistical inference on a sample of 'voluntary' subjects, opposed to randomly selected subjects. Basic statistical inference rests on the assumption that a population sample is a random and representative selection from the population proper. If subjects are voluntary, then you cannot be sure that a voluntary patient is truly representative of the population. For example, a patient may be a more willing volunteer for the survey if their medical treatment was successful. Therefore your sample, and inferences therefrom, are statistically biased.

The basic premise of the statement appears to be that samples must be representative in order to 'generalize' statistical conclusions.

-

The difficulty is you can't normally assert that a convenience sample such as a set of volunteers will tend to have the same characteristics as the population you wish to generalize to. Indeed convenience samples generally tend to be distinctly unrepresentative.

Even if you were comparing two treatments and had random assignment of volunteers to treatment, the response of the particular subset of the population likely to volunteer would not necessarily be like that of the general population (though in some cases you may be able to make an argument that there should be no relationship between the difference in treatments and propensity to volunteer - keeping in mind that such things as age and employment status can be factors in volunteering for many studies, which might make such an argument difficult in many cases).

-

I asked the same question on the discussion forum of OpenIntro(Stat Book), Here is an answer I received from David,

The primary concern is that there might be something different about people who volunteer vs people who do not volunteer. The best way to think about this type of problem, in my opinion, is to play Devil's Advocate. Below are hypothetical examples of potential concerns about generalizing the results for this study. I have not recently looked at the study details, so the authors may address some of these concerns within the corresponding paper.

Example #1. If the study was isolated to local volunteers living in the same town as the researchers, it is possible that these patients share common characteristics that make them more receptive to the Buteyko method than people living in other regions.

Example #2. Perhaps people with only a specific severity of asthma (say, a mild form of asthma) might have felt comfortable volunteering for such a study. In this case, the method may work for patients with a mild form of asthma but ineffective or even harmful for patients with a severe form of asthma who would not be measured by the study.

These are just two examples of "what ifs" for this study. It would be worth taking a few minutes to think about other concerns that might complicate generalizing the study results to a large population.

On the other hand, if the patients were from a random sample of the population considered, this would generally mitigate these "what if" concerns.

-
This answer corresponds to everything that has already been said in this post. In particular, see the wikipedia article on selection bias that was referenced above. – gregory_britten Jul 3 '14 at 18:46