1
$\begingroup$

I have a study. I have 150 AI-generated images and 150 human-generated images. I have created a survey where respondents are given 10 images randomized from these 300 images. For each picture, they are asked if they think it's human-generated or AI-generated. I want to see if there is a statistically significant difference in the respondents' ability to distinguish between AI-generated and human-generated memes. How should I go about calculating how many respondents I approximately need for this?

Specifically, I wonder if the fact that the respondents are shown only a subset of the entire image set affects the sample size I need. I understand that the size then depends on desired significance level, power level, and effect size.

Improve this question
$\endgroup$
3
  • $\begingroup$ What statistical test are you thinking of using? $\endgroup$
    – Sointu
    Commented May 22, 2023 at 11:52
  • $\begingroup$ I assume a Chi-2 test where I try to see if there is a significant difference between random guessing and the actual outcome $\endgroup$
    – perceptronEnthusiast420
    Commented May 23, 2023 at 6:55
  • $\begingroup$ OK, in that case you can for instance use pwr.chisq.test function from the pwr package in R. In my understanding, in your design the sample size provided by pwr would refer to the number of trials, so if each participant judges 10 images, the number of participants would be n/10. However you might need to use an analysis that takes into account that same people are rating many images, and that same images are rated several times (such as multilevel logistic regression), in which case power calculations need to be conducted via simulations (e.g. with simr package). $\endgroup$
    – Sointu
    Commented May 23, 2023 at 7:23

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.