Statistical Power of ROC/AUC Test with non-IID Samples :: To how many IID Samples are my non-IID Samples Equivalent? I've been assigned to solve the following problem as part of a serious, biological research project. I think I have a tentative solution, but I'm wondering whether the approach I've picked is the best. Your help/input would be appreciated.  
We monitored the vital signs (V) of ten subjects as they travelled up and down a mountain. From each subject, we took approximately 100 samples, one every minute --- fifty when they were at the top, and fifty at the bottom of the mountain. A sample contained a measurement of five vital signs of the subject (respiration rate, heart rate, etc.).   
Along with each sample of vital signs, we also recorded whether the patient was on the top (T) or on the bottom (B) of the mountain.  
For each sample, we computed Stress (S), where S is a function the vital signs, so S = f(V). (The computational details of f are not relevant.) 1000 samples were drawn from 10 subjects, whereby we had 1000 values for Stress, each of which was coupled to a piece of binary information (B or T). 
The purpose of collecting the binary information is to test our (previously written) binary classifier: based on a Stress-value, the classifier decides whether or not the subject is on the top or bottom of the mountain. By comparing this to the actual binary value (top / bottom), we can populate a confusion matrix. 
We then iteratively vary the discrimination threshold of the binary classifier, to finally construct an ROC curve. This gives us the Area Under the Curve (AUC), which we use as to assess the quality of our classifier. That is the end result we are interested in. 
The important question has been apparent from the start: what's the statistical power of our end result (the AUC), given that we're really only sampling from 10 subjects? More concretely, our samples are obviously not IID. We're asking ourselves: approximately how many IID samples are our 1000 samples equivalent to? We have been thinking about the power of our test with 1000 non-IID samples as equivalent to a test with $n$ IID samples and the same power. Is there any statistical methodology that will let us find $n$? This is the general direction of the question I'm trying to answer.
My tentative solution involves the use of General Estimating Equations. A similar approach seems to have been taken by the authors of the CLEPSYDRA study (available here). My reasoning is that the covariance structure of the samples is essentially unknown, which is a problem that GEEs would let me overcome. But I'm not sure if there won't be further complications, i.e. if this is really the right tool for the task, or if there isn't a simpler approach. Is there a simpler approach? Are there further issues that using a GEE won't solve?
 A: This is a power calculation that can only be achieved via simulation. Assuming that you are fine with the simulation components of the analysis up to simulating correlated binary outcomes, I will attempt to address that deficiency. It is usually reasonable to simulate correlated categorical outcomes using a random effect in the linear component of the fitted model. For instance, if you're an R user, this might look something like:
npers <- 100 # number of persons in sample
nmeas <- 10 # number of measurements taken on each person
id <- rep(1:npers, each=nmeas) # id variable for the "long" data (1000 rows)

# variance of person level random effect
vpers <- 2 

# actual random effect
raneff <- rep(rnorm(npers, 0, vpers), each=nmeas) 

# number of fixed effects in linear model
np <- 4 
x <- cbind(1, matrix(rnorm(npers*nmeas*np), npers*nmeas, np)) # design mx
b <- matrix(c(-4, rnorm(np)), np+1, 1) # randomly generate fixed effects
nu <- x%*%b + raneff # combined fixed and random effects in linear predictor
eta <- plogis(nu) # conditional mean

# randomly generate outcome
y <- rbinom(npers*nmeas, 1, eta)

library(lme4)    

## unconditional model effects biased toward zero from noncollapsibility
glm.fit(x, y, family=binomial())$coef 

## better for individual level effects
glmer(y ~ x[, -1] + (1|id), family=binomial)@fixef 

The major caveat with, say, risk prediction from repeated measures designs is the following: the fitted values from fixed effects in a conditional (mixed) model predict outcomes in a person whose random effect is exactly zero. Remember random effects arise from millions of unmeasured prognostic factors. If you actually measured all those factors, there would actually be no need for random effects because repeated measures would be conditionally independent from one another since you've controlled for every possible level of variation within and between persons.
However, and I think this is a somewhat unexplored principle, I think you can get better risk prediction from marginalizing the distribution of conditional fitted risk values according to the estimated distribution of the random effect. This is technically a better approximation to whatever the actual risk is relative to the GEE. The GEE estimates the logit of the average whereas a marginalized mixed model would give you the average of the logit. Jensen's inequality gives us that the average of the logit is not the logit of the average. Which is right? Remember, it's the particular individuals' random effect which is unknown. Hence you are averaging up a bunch of possible logits to estimate their risk. We showed something similar with missing data and risk prediction before.
