I have a sample of $N=30$ questions that the test subject is going to rate as either True (T) or False (F). The subject is informed that the ratio of T/F answers is exactly 50%. We can assume that the subject will adjust his answers so that he/she will meet the 50% ratio (and thus would never score 17 T and 13 F).
Question: What would be the correct test to reject the null hypothesis that the subject can't distinguish between the two classes with a $p=0.05$?
Thoughts: There are $30!/(15! 15!) \approx 1.5*10^8$ different permutations of the 15(T) and 15(F). One could work out the number of ways to get exactly $k=0,2,4,...,30$ correct and from there construct a test. I have a feeling this problem has already been solved before and has a name... I am interested for the specific case and the general case when $N$ is arbitrary and the ratio differs from 1/2.