Test to determine if a set of coins have specified biases I'm playing a game with a friend where we first select a random coin from a bag. This coin has a bias $q$ chance of coming up heads and my friend knows what $q$ is when he selects the coin, and tells me "q is (some number)", but may be lying to me. I don't know what $q$ is.
I then choose heads or tails and win if I am correct.
We play this game $N$ times, each time choosing a different coin with a different $q$. However, I suspect that my friend is cheating, eg. that the value of $q$ is not the value he claims it to be. 
Thus I have a sequence of values:
\begin{align}
Q &= [q_1, q_2, q_3, ..., q_N] \\
R &= [T, T, H, H, ..., T]
\end{align}
How can I construct a statistical test (modified version of the binomial test?) to reject the null hypothesis that my friend is not cheating from my observations?
Is this a known variant of the binomial test?
 A: This problem is really similar to that of calibrating the output of a machine learning binary classifier. The output $Q$ is (supposedly) the vector of probabilities of getting heads, and if he is telling the truth it should be exact. So, perform logistic regression of the outcomes $R$ (1 for heads and 0 for tails) as a function of $\log(Q)-\log(1-Q)$. If your friend is telling the truth, then the intercept should be consistent with $0$ and the slope consistent with $1$. Of course, these parameters are correlated, but you can use the covariance matrix to test if the parameters are consistent with $(0,1)$, yielding a score from a $\chi^2$-distribution with two degrees of freedom.
Of course, you can also do a non-parametric test; you can consult many texts on logistic regression to find candidates (e.g. Hosmer-Lemeshow test; Pearson and le Cessie-van Houwelingen-Copas-Hosmer test, etc.). Some of these are implemented in an R package, and I'll add a mention of it when I remember its name. However, actually performing logistic regression should work just fine.
EDIT
The rms package provides parametric goodness-of-fit tests for the result of a logistic regression via the residuals.lrm function. There are parametric tests done after performing logistic regression, and have some power to reject cases where the linear hypothesis isn't as good as alternative non-linear hypotheses (though they aren't perfect). Here is an example showing the le Cessie-van Houwelingen-Copas-Hosmer test:
library(rms)
dllq <- rnorm(1000,1,2)
q <- 1/(1+exp(-dllq))
y <- sapply(q,function(qi){rbinom(1,1,qi)})
fit <- lrm(y~dllq,x=TRUE,y=TRUE)
print(fit)
residuals(fit,type="gof")

The output of this may look like the following:
Logistic Regression Model

lrm(formula = y ~ q, x = TRUE, y = TRUE)

                      Model Likelihood     Discrimination    Rank Discrim.    
                         Ratio Test            Indexes          Indexes       
Obs           100    LR chi2      44.51    R2       0.493    C       0.867    
 0             36    d.f.             1    g        2.370    Dxy     0.734    
 1             64    Pr(> chi2) <0.0001    gr      10.697    gamma   0.735    
max |deriv| 1e-06                          gp       0.339    tau-a   0.342    
                                           Brier    0.145                     

          Coef    S.E.   Wald Z Pr(>|Z|)
Intercept -0.0579 0.2852 -0.20  0.8390  
q          1.0072 0.2131  4.73  <0.0001 

Sum of squared errors     Expected value|H0                    SD                     Z 
           14.4577468            14.0041517             0.3717001             1.2203253 
                    P 
            0.2223416 

So, the p-value is 22% which is more than comfortably within the "good fit" region. In combination with the compatibility of the intercept and slope with 0 and 1, this means "actual heads probability consistent with q: no evidence of lying."
I haven't yet found a "non-parametric" version of this test in R, which skips the logistic regression (since your data should already be calibrated) and which I think would be the best test in this case.
SECOND EDIT
I've had no luck finding a native R package that implements non-parametric versions of the standard binomial regression goodness-of-fit tests. However, it's not too hard to calculate them manually. There are three quantities to choose from:
$$
G^2 = \sum_i R_i \log(\pi_i) + (1-R_i)\log(1-\pi_i)\\
X^2 = \sum_i  \frac{(R_i-q_i)^2}{q_i(1-q_i)}\\
S = \sum_i (R_i-q_i)^2
$$
The first is the deviance, the second the Pearson statistics, and the third has a very long name I've mentioned a couple of times. Each one has been shown to asymptotically follow a normal distribution, if the probabilities $q_i$ are accurate predictors of $R_i$ (see Osius and Rojek). All you have to do is calculate their means and variances. For example,
$$
E[S] = \sum_i \left( E[R_i^2] - 2 E[R_i] q_i + q_i^2 \right) = \sum_i q_i(1-q_i) \\
E[(S-E[S])^2] = \sum_i E \left[\left( (R_i-q_i)^2 - q_i(1-q_i) \right)^2 \right] + \sum_{i \neq j} 0 = \sum_i q_i(1-q_i)(1-2q_i)^2
$$
Once you figure these out, you can easily calculate them in R and get z-scores. The three z-scores you get aren't independent, but these different tests are sensitive to different types of errors in the probabilities $q_i$.
