Designing a test for a psychic who says he can influence dice rolls Let's say that I have a friend (let's call him "George") who says that he can control the roll of dice using his mind (ie, make the dice more likely to fall on a specific number he is thinking of). 
How can I design a scientifically rigorous test to determine if he can actually do this? (I don't really think he can, of course, but I want him to agree to the details of a test, Amazing Randi-style, before the test begins.) I want to reduce the (very likely) post test-excuses that he'll come up with.
Here is what I have so far: 


*

*Determine the physical dice-rolling technique (which dice, shaker cup, landing surface, etc)

*Define a "test session", consisting of X rolls of the dice. This needs to be small enough to do in one sitting, but large enough to determine (after analysis) within 95%-99% confidence whether the dice fell fair, or favored one side

*Run Y sessions on the chosen dice (with no influence from George), as a "control" to make sure that the dice show "fair" results on their own

*Run Z sessions with George. Before each, roll a separate die to determine what number George will "concentrate on" during that entire session.

*Compile and analyze the results.

*George makes up some excuses for his dismal performance.
So my questions for you:


*

*Any flaws or problems with my overall methodology? Anything George would likely object to?

*Should I use a D6? Or a D20? Does it matter? Would a die with more faces require more rolls to produce similarly confident results? Or the opposite? I'd prefer fewer rolls than more, due to practical considerations :)

*What are reasonable values for X, Y, and Z? They are not entirely unrelated; if my chosen value of X only allows 95% confidence for a single session, then 1 out of every 20 sessions could "fail", even without George's influence

*How do I define "success" or "failure" for an individual session? (I did find this question which goes through the details of a chi-squared test, so I think that's my evaluation methodology, but what are reasonable confidence thresholds?)

*How do I define "success" or "failure" for the overall test? George might "win" a single session by sheer chance, but how many of the Z sessions would he have to pass to pass the whole test?
I'll probably be analyzing these results in a MS Excel spreadsheet, if that makes any difference.
 A: I would recommend analyzing this in the following way:
Count each role in which George successfully predicts the outcome as a success, and every other one as a failure.  Then, you easily calculate a probability of success for George, and a 95% or 99% confidence interval.  Does he claim he can predict the outcome "twice as well" as randomly rolling the dice?  Then:
H0: p >= 1/3
H1: p < 1/3
(assuming a 6-sided die).
From there, it's pretty straightforward to do the hypothesis test.  Also, you can calculate the power a priori pretty easily (even in something like Excel).  Pick a number of rolls (like 10), and then make a table with the possible successes as rows (0-10).  Then, for each success, calculate the probability he'll have that many successes (if he were to be just guessing, which is what we're assuming he is doing).  Also, for each value, determine if it would lead to a rejection or acceptance of the null.  Then, to find the power, you can simply add up all the probabilities where the null would be rejected.
A: A D20 will require more rolls for the same significance level for George succeeding for sure if you need to run the chi-square test. I don't think you need to run the full chis-square test though. You just need to check whether the dice rolls the "chosen" number more often than chance. I would just use the cdf of the binomial to calculate the p-value of rolling the chosen number more often than chance with $\theta=\frac{1}{6}$ as the binomial parameter for D6. I think it is easy to determine the number $X$ based upon the p-value you need for George succeeding. I am not even sure you need Z sessions. Why not just run one session for each side of the die. Does randomizing the chosen side even matter for the hypotheses you are interested in?
