1
$\begingroup$

I am thinking of using a company that has an online panel to recruit participants for a questionnaire study. Because the demographics (age, gender, region etc) of the entire population is known (population is around 30,000), they say that they can claim representativeness of the results as they can ensure the demographics of the sample matches the population. Does this sound right?

(sample would be at 95% confidence levels with 5% confidence interval).

Improve this question
$\endgroup$
5
  • $\begingroup$ Could you elaborate on what it means to "have an online panel"? Are you asking whether recruiting participants entirely via online communications will produce a representative sample provided the "demographics" of the sample "matches" the population in some sense? (I use quotation marks to indicate dubiousness: there are infinitely many possible demographic characteristics and perfect agreement of sample and population will be impossible, making this vague criterion of representativeness highly suspect.) $\endgroup$
    – whuber ♦
    Commented Mar 3, 2015 at 15:22
  • $\begingroup$ Perhaps we need a bit more information on this. $\endgroup$
    – Analyst
    Commented Mar 4, 2015 at 8:00
  • $\begingroup$ @whuber Thank you for replying. The sample and method used would be same as this paper: link (We surveyed 1715 general practitioners and 783 (46%) completed our questionnaire. Our respondents were similar to those of all registered UK doctors suggesting our results are generalizable. ) $\endgroup$
    – CakeNRun
    Commented Mar 4, 2015 at 12:03
  • $\begingroup$ @whuber Online panel means that a certain percentage (around 15%) from the entire population have agreed to take part in research, and participants from this group will be recruited to take part in the research. $\endgroup$
    – CakeNRun
    Commented Mar 4, 2015 at 12:12
  • $\begingroup$ You should expect this panel to have unrepresentative opinions: people who agree to participate in such studies tend to differ (in some ways dramatically) from those who do not. If you further trim it down to match various demographic characteristics, the lack of representativeness will not thereby be fixed. $\endgroup$
    – whuber ♦
    Commented Mar 4, 2015 at 15:27

2 Answers 2

Reset to default
1
$\begingroup$

The sample itself is not representative. But you could randomly select on demographic characteristics to meet those of the general population. Or otherwise attempt to control for your bias. In short: the selection is biased but there exists some methodologies that will help mitigate the problem.

Improve this answer
$\endgroup$
0
$\begingroup$

In principle yes but there are possibilities that participants are special case where some sort of unobserved selection mechanism is at work.

Improve this answer
$\endgroup$
4
  • $\begingroup$ In a single sentence you appear to say both "yes" and "no"! If there is a real possibility of selection effects, doesn't that mean even in principle it would be invalid to claim the sample is representative? $\endgroup$
    – whuber ♦
    Commented Mar 3, 2015 at 15:23
  • $\begingroup$ @whuber devil is in details. When do not know how this panel is formed. I could imagine situation where using it and making inferences could be valid, but there is a big possibity of biases. $\endgroup$
    – Analyst
    Commented Mar 4, 2015 at 8:00
  • $\begingroup$ @Analyst Thanks for replying. The sample and method would be like the one taken in this paper: link (We surveyed 1715 general practitioners and 783 (46%) completed our questionnaire. Our respondents were similar to those of all registered UK doctors suggesting our results are generalizable. ) $\endgroup$
    – CakeNRun
    Commented Mar 4, 2015 at 12:07
  • $\begingroup$ The "suggestion" is a weak one indeed. The authors merely hope there is no association between electing to complete the questionnaire and the physicians' experience with placebos. In your case, if the questionnaire is about the company itself, then that hope dies: one would fully expect the self-selected panel to have certain biases related to their willingness to participate. $\endgroup$
    – whuber ♦
    Commented Mar 4, 2015 at 15:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.