I would like to compare the effect of a treatment between two groups.

My dataset is summarized in this table:

enter image description here

Each score corresponds to a proportion (namely, the proportion of successes on a given cognitive test).

How would you compare the difference of treatment effects between Group 1 and Group 2, taking into account the within-subject variability? Would you use a logistic regression?

Thanks a lot


If your interest is in overall test scores, rather than the individual answer probabilities, you probably want to consider your outcome variable $score_{\text{After}}-score_{\text{Before}}$. However, if you want to consider individual question probabilities nested within subjects within disease groups, you would want to explicitly model each question's paired difference separately (although it seems like different subjects had different numbers of tests… is there a missing information issue here?).

You have an N = 3 subjects, which seems pretty challenging to base much in the way of inference on (especially given that all subjects do not share all disease groups). You also will not be able to separate the effect of being Subject A from being in disease group A since there is perfect overlap with these identifiers. Likewise, because there are only two subjects in disease group B, you are going to have a very difficult time parsing out whether the differences results from disease group differences or subject differences. Interaction effects seem a far cry from what these data can support, if I understand them correctly.

Without understanding the particulars of your study design it is likewise difficult to say much about the nature of your inferences with respect to causal evidence.

I realize this answer is not a solid "here's what you need to do", but since you contacted me via email to request my attention, I thought I would provide you with my initial impressions.

  • $\begingroup$ Thanks for your feedback. The duration of each test is fixed, but the number of trials (each of which results to "success" or "failure") is determined by the subject (s/he initiates each trial). That's why the number of trials is different before and after treatment for a given subject; "paired subjects" without "paired data"? My interest is to compare the treatment effect between group 1 and group 2. Supposing there are many more subjects in each group, I would be curious to know how you would handle this problem. $\endgroup$ – Salomon Feb 19 '18 at 0:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.