Take the 2-minute tour ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

I have to create a questionnaire that compares the sportiness of Facebook users vs. non-Facebook users. I need to calculate the number of participants that are required so that the result is statistically relevant. Unfortunately I have no idea how I can calculate that. Could you help me?

share|improve this question
add comment

4 Answers 4

Defining the problem

  • I am guessing that you are measuring sportiness on some form of multi-item scale. Thus, you will have a numeric measure of sportiness. I am assuming that you will be running an independent groups t-test to test the difference between group means.

  • Your problem is then a form of a priori power analysis. To determine the sample size you need to specify:

    1. Your desired statistical power (often 80% is considered adequate)
    2. Your expected effect size (typically specified as standardised group mean difference; of course this is not known ahead of time, so you have to think about what is the minimal effect that you would consider interesting)
    3. Your alpha level for your significance test (typically .05)

Software options

  • A nice free GUI for calculating statistical power is G Power 3 available on Mac and Windows. I have an explanation of basic use with some sample power analysis graphs relevant to your example (standardised group mean differences).

  • R also has options as summarised on the quick-r page on power analysis. Check out pwr.t.test in the pwr package for one option.

  • The basic rule is that more participants is always better, and that any power analysis rests on assumptions about population effect which are unknown (if they were known, you wouldn't need to do a study).

R Example:

> install.packages("pwr")
> library(pwr)
> pwr.t.test(power= .8, d=.5, sig.level=.05, type="two.sample")

     Two-sample t test power calculation    
              n = 63.76561
              d = 0.5
      sig.level = 0.05
          power = 0.8
    alternative = two.sided    
 NOTE: n is number in *each* group 

Thus, assuming what is often labeled as a medium difference in group means and conventional values for alpha and power, you would need 64 participants per group.

Graph from G Power

The following graph was generated from G Power and is taken from my post on power analysis.

It shows for different levels of d, what power you will obtain for a given total sample size.

graph of power analysis for different d

Final complication if non-experimental design

The above calculations are all done on the assumption that you have equal numbers of facebook and non-facebook users in your sample. This feature is common to experiments and to studies where participants are sampled in some systematic way. However, if you are just taking a general sample from the community, you will end up with uneven numbers of facebook and non-facebook users.

All else being equal statistical power decreases as group sizes become less equal. G Power 3 allows you to specify the ratio of group sizes when calculating required sample size.

share|improve this answer
add comment

Here is a link to a website that do that for you, but it is in german... :-\ But mybe you can use it...

http://www.bauinfoconsult.de/Stichproben_Rechner.html

This site calculates the number of needed respondents according to the standard error and the expected power.

share|improve this answer
1  
Welcome to our site, @Dominik! Even when references are in English, we prefer that you summarize their contents in your reply: this makes the thread self-contained and long lasting. Perhaps you could edit your reply to provide this information. :-) –  whuber Aug 1 '11 at 14:04
    
Google Chrome has a nifty translate feature that could be of use. –  Chase Aug 1 '11 at 14:39
    
Hi whuber, sorry, I'll do it next time. for now I just updated my answer... But the answer from jeromy is much better! –  Dominik Aug 2 '11 at 15:59
add comment

Here's an English version

Questionnaire or survey sample size How do you work out what sample size to use for your survey? It is actually a complex calculation. And consequently, in my experience, people use rules of thumb - like 10%. Such rules of thumb cannot hope to give an adequate estimate of the needed sample size and consequently people either under-sample or over-sample. Often these samples are vastly too small - causing inappropriate decisions - or much too large - a wasted effort and expense.

What sample size should you take? The answer is a balance between your intolerance for 'false positives' and 'false negatives'.

You can also try the Sample Size Calculator at gpra.net - here's the Intro and Install PDFs

share|improve this answer
    
That was exactly what I was looking for. The only thing still missing is the formula. Is it available somewhere? I couldn't find it. I assume as population size I have to enter roughly 6 billion (every human), right? –  Markus Lanthaler Aug 1 '11 at 20:00
    
@MarkusLanthaler How are you going to randomly sample from the 6 billion (actually, it's well over 7) people on the planet? Are you really going to sample Somalia and Brazil and Vietnam in proportion to their populations? From your question this doesn't sound like your target population at all, which sounds more like "people who could use facebook but may or may not actually do so". –  Glen_b Aug 28 '13 at 2:33
add comment

Lehr's rule, as quoted by Van Belle is $$n = \frac{16}{\Delta^2},$$ where $\Delta$ is the posited effect size, which in your case would be $(\mu_{\mbox{fb}} - \mu_{\mbox{non fb}}) / \sigma$, where $\mu$ is the mean 'sportiness', and $\sigma$ is the (pooled) standard deviation of sportiness. You want to collect $n/2$ participants from Facebook and the remaining half not from facebook.

This rule gives you approximately 80% power for a 2-sample 2-sided t-test at the 0.05 type I rate.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.