My question is about RCTs where we receive many repeated observations per unit, but we don't necessarily want to aggregate them together. A concrete example is as follows.
Suppose I am running an RCT for two versions of a studying tool for students. Suppose we don't know in the students homeworkreal world. Each student reports back their grades on homework assignments, but we don't know in advance how many observations they will report.
The hypothesis is that the #treatment group will increase their homework scores (say, from 70% to 75%). I have student covariates at the time of assignments is randomtreatment assignment (e. For example:g., GPA, year in school, etc), which I plan to use in the regression.
Treatment | Student id | Observations |
---|---|---|
1 | 1 | 90/100 (subject 1), 8/10 (subject 3), 7/10 (subject 5) |
1 | 2 | 80/100 (subject 2) |
1 | 3 | 40/50 (subject 4), 15/20 (subject 2) |
0 | 4 | 70/100 (subject 2) |
0 | 5 | 7/10 (subject 1), 40/50 (subject 3) |
0 | 6 | 7/10 (subject 5) |
Intuitively, I would like to say that each student has some probability $p$ of answering a question correctly. I have student covariates at the time of treatment assignment (e.g., GPA, year in school, etc), which I plan to use in the regression.
A few specific questions:
- One way to do this is to aggregating all of the data together by summing numerator and denominator. Now each student has a single observation (for student 1, it'd be 105/120), but this seems like I wouldn't be missing the full information I have. Is there a way to design a regression setup that can help me analyze this data without aggregating? Should I use student random effects in addition to the student covariates?
- Suppose the students also report the subjects that the homeworks were for. Since these are observed after treatment assignment, they shouldn't be used as covariates in a regression model. Is there any way to use this information? Because the observations can be for different subjects, there will be some variation in the observations which I'm not sure how to handle.