1
$\begingroup$

I'm an honours student in psychology doing some data analysis at the moment. My research project is coming to a close for data collection, and it seems like we won't reach our initial goal. I decided to have a look at g-power to determine how much statistical power my results would have, but I'm experiencing a lot of confusion. To put it simply, my research involves a simple condition/control pre-post treatment analysis. I'm using R to perform mixed model ANOVAs and mainly interested in the interaction (of time*condition).

In G-power, I'm using the F tests, Anova: repeated measures, within-between interaction option. Assuming that the effect size f input parameter means Cohen's f (where .10 is a small effect, .25 is a medium effect, and .40 is a large effect), I input the parameters as follows and obtain the following result for a small effect size: enter image description here

I then change my parameters to estimate the required sample size for a large effect: enter image description here

Can someone please explain to me why it seems like I'd require a smaller sample size to obtain a larger effect? I'm clearly not understanding something here and it's very frustrating haha.

Many thanks!

$\endgroup$

3 Answers 3

2
$\begingroup$

Be careful with doing sample size calculation using G*Power for this design! You need to take into account that the effect size "f" is specified in terms of a "double dissociation effect". See the discussion in: GPower: Difference in Sample Size for ANCOVA vs. Repeated Measures ANOVA in clinical trials

$\endgroup$
1
$\begingroup$

I think the best way to illustrate this is by example.

Let's say you want to see if American men are taller than American women. How many people will you need to be reasonably sure of your answer?

Now, suppose you want to see if Norwegian men are taller than American men. Will you need more people than in the first example or fewer people?

Now suppose you want to see if basketball players are taller than jockeys. Will you need more people than in the first situation or fewer people?

The effect size in the first situation is moderate. In the second, it is small. In the third it is huge.

$\endgroup$
8
  • $\begingroup$ I’m still confused haha. I understand what you’re saying, but don’t necessarily understand how it translates to sample size estimates. Are you saying that it’s always the case that you’d expect to find a large effect size when examining a small sample? $\endgroup$
    – Amber
    Aug 24, 2019 at 10:57
  • $\begingroup$ Is it related to larger samples regressing towards the mean? So the difference between groups would be smaller? $\endgroup$
    – Amber
    Aug 24, 2019 at 11:00
  • $\begingroup$ No, no. It's a question of how big a sample you need in order to find an effect. You need a bigger sample to find a smaller effect. $\endgroup$
    – Peter Flom
    Aug 24, 2019 at 11:12
  • $\begingroup$ I guess I’m just confused because I’ve always thought a larger effect size was better/more important/more powerful, so I assumed you would require a larger sample. Is that not the case? $\endgroup$
    – Amber
    Aug 24, 2019 at 11:36
  • 1
    $\begingroup$ Nope, it's not the case. $\endgroup$
    – Peter Flom
    Aug 24, 2019 at 15:03
1
$\begingroup$

In your question, you state: Can someone please explain to me why it seems like I'd require a smaller sample size to obtain a larger effect? I'm clearly not understanding something here and it's very frustrating haha.

The effect you are trying to estimate is not something you can obtain. The effect is an unknown quantity which refers to the underlying population(s) you are trying to learn something about. The effect assumes a unique value - it's just that you don't know what it is.

In the first example provided by Peter, the effect of interest could be defined as the difference between the average height of American men and the average height of American women. While this difference is unknown to you, it assumes a unique value. (The only way to find out what this unique value is would be to measure the heights of all American men and the heights of all American women, compute the average height for each sex and then take the difference of the two average heights. This unique value is called the true difference between the average height of American men and the average height of American women.)

When you perform a sample size calculation for this particular example, you acknowledge right off the bat that you will be be unable to measure the heights of all men and all women in the US. Instead, you are going to take a random sample of n men and a random sample of n women from the US population. The question is: How big should n be to ensure that you are able to detect (on the basis of a test of significance) a true difference between the average height of American men and the average height of American women of at least the magnitude you pre-specify based on subject-matter knowledge?

Since you don't really know what the true difference is, all you can do is venture that the true differece is small, medium or large in size. For any given study, only one of these claims can be true, so you'll have to do your best to choose which claim is most suitable for that study (e.g., conduct a pilot study, reference similar results in the literature). See https://academic.oup.com/ndt/article/25/5/1388/1842407 for more details.

It is easier to detect true differences (or effects) that are postulated to be large than those that are medium or small in size.

$\endgroup$
1
  • $\begingroup$ @Amber: It seems that Peter has already answered your question. $\endgroup$ Aug 25, 2019 at 14:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.