In our office, I have been involved in a discussion about sample size and its influence on effect size - can you help me out and explain further?
When conducting a power analysis, one can determine sample size for a specific effect size in a specific design.
What happens if the a priori determined sample size is exceeded (e.g. determined sample in power analysis was $N=100$, but we could obtain $N=1000$)?
Position 1: Large sample sizes chop up/ destroy effect sizes. When using larger samples than determined in power analysis, danger occurs that "everything becomes significant" (even minor, practically irrelevant effects). Therefore, we should rely to determined sample from power analysis. Doing so, we can reveal "real/relevant" effects.
Position 2: Determination of sample size is referring to the minimal sample size which is required to reveal a given effect. Large sample sizes are beneficial, e.g. because of decreasing measurement error. Therefore, real effects can be revealed more easily. Post hoc effect-size calculations offer information about relevance of the effect.
Position 3: Position 1 versus position 2 are depending on the study design (e.g. position 1 for t-Test because of seeking for "relevant effects", but position 2 for CFA/ SEM to get more stable, reliable results).
Position 4: Another possible position for an alternative explanation.