I've done an experiment where I've collected measurements from a number of participants. Each relevant data point has two variables, both categorical: in fact, each variable has two possible values (answers to two yes/no questions). I would like a statistical hypothesis test to check whether there appears to be a correlation between these two variables.
If I had one data point per participant, I could use Fisher's exact test on the resulting $2 \times 2$ contingency table. However, I have multiple data points per participant. Consequently, Fisher's exact test does not seem applicable, because the data points from a single participant are not independent. For instance, if I have 10 data points from Alice, those probably aren't independent, because they all came from the same person. Fisher's exact test assumes that all data points were independently sampled, so the assumptions of Fisher's exact test are not satisfied and it would be inappropriate to use in this setting (it might give unjustified reports of statistical significance).
Are there techniques to handle this situation?
Approaches I've considered:
One plausible alternative is to aggregate all the data from each participant into a single number, and then use some other test of independence. For instance, for each participant, I could count the fraction of Yes answers to the first question and the fraction of Yes answers to the second question, giving me two real numbers per participant, and then use Pearson's product-moment test to test for correlation between these two numbers. However, I'm not sure whether this is a good approach. (For example, I worry that averaging/counting is throwing out data and this might be losing power, because of the aggregation; or that signs of dependence might be disappear after aggregation.)
I've read about multi-level models, which sound like they are intended the handle this situation when the underlying variables are continuous (e.g., real numbers) and when a linear model is appropriate. However, here I have two categorical variables (answers to Yes/No questions), so they don't seem to apply here. Is there some equivalent technique that is applicable to categorical data?
I've also read a tiny bit about repeated measures design on Wikipedia, but the Wikipedia article focuses on longitudinal studies. That doesn't seem applicable here: if I understand it correctly, repeated measures seems to focus on effects due to the passage of time (where the progression of time influences the variables). However, in my case, the passage of time shouldn't have any relevant effect. Do tell me if I've misunderstood.
On further reflection, another approach that occurs to me is to use a permutation test. For each participant, we could randomly permute their answers to question 1 and (independently) randomly permutation their answers to question 2, using a different permutation for each participant. However, it's not clear to me what test statistic would be appropriate here, to measure which outcomes are "at least as extreme" as the observed outcome.
Related: How to correctly treat multiple data points per each subject (but that also focuses on linear models for continuous variables, not categorical data), Are Measurements made on the same patient independent? (same)