Let me just start by saying that I am no statistician and might be misinterpreting some concepts, and appreciate all the help I can get with this problem. I creating a survey, in which participants are to evaluate short video clips (~5sec) generated by a machine learning algorithm. The videos show dashcam footage from a traffic simulator and are one of two possible image types (RGB images or semantically segmented images). I would like to investigate the effect of using one image type over the other when it comes to how realistic-looking videos the algorithm generates (i.e. evaluation of an unsupervised generative model).
So now comes the actual questions. My null hypothesis is that there is no difference in the degree of realism between the groups (image types). I am trying to figure out how many participants I need for my survey, which from my understanding is what's called the sample size. In order to calculate the sample size, I need to know the population size, but here is where I get confused. Who or what is my population? The only restriction I have on my participants is their age which must be 18-80 years. Or am I misinterpreting what a population is, maybe it should rather by my total distribution of videos? Have I already chosen a sample size, which is the number of videos on my experiment? Please let me know if I am on the wrong track here.
There are in total 104 videos in the survey, whereof 26 are generated RGB, 26 are ground truth (label) RGB, 26 are generated segmented and 26 are ground truth segmented.
At this point, my thoughts are to use a t-test with a significance level of 0.05, effect size Cohen's d of 0.8 and statistical power of 0.8. But of course, I am open to suggestions.
Please let me know if I should elaborate further on my problem. Thank you!