I have performed an experiment, where I have real and synthesised movie data. My participants were shown either a real movie, or a synthesised movie (of talking heads), in a random order, and had to guess whether the movie was real or synthesised.
I got a binomial test result of 0.58 (using MATLAB
binofit). Of 2040 samples, 1189 were guessed correctly which I think shows significant difference.
I also tried McNemar's test. The confusion matrix is:
[506 359] [449 582]
where top left is 'said synth when synth', top right is 'said synth when real', bottom left is 'said real when synth' and bottom right is 'said real when real'.
So anyway, McNemar's test for that gives me 9.91, which again I think shows that there is a significant difference.
Are there any other tests I can try?