How to compare before and after evolutions of a binary variable between two groups of treatment

Question

Here is what look like my data

Groupe <- c(rep("A", 7), rep("B", 7))
    
DiabeteBeforeTreatment<-c(
      "no", "yes", "yes", "yes", "no", NA, NA, 
      NA, "no", "yes", "yes", "yes", "no", 
      "yes")
    
DiabeteAfterTreatment <- c(
      "no", "no", "no", NA, NA, "yes", "no", NA, 
      "yes", "yes", "yes", NA, "no", "yes")   
 
mydata <- data.frame(Groupe, 
     DiabeteBeforeTreatment, 
     DiabeteAfterTreatment)

I want to compare whether one treatment improves diabetes better than another treatment. I have mixed cases before the treatment started: Some are diabetic and some are not. After treatment, patients may have different outcome possibilities, either they get better from their diabetes, they stay the same, or they get worse, i.e. they go from not having diabetes to having diabetes.

I am wondering what kind of statistical test I could use to compare the two treatments and determine whether one treatment improves the patient's condition better than the other.

Do I need to take into account the matching of the data in the analysis? Then I use a McNemar test. In this case, I would only do the analyses on patients who have before and after values (as I do not intend to impute).

Or I could use all patients in the analysis and calculate the number of yes and no in each group, after and before treatment, and compare the effectiveness between the groups without considering matching. I will only compare proportions of yes a each period between the treatment groups

I thought of another method, but only involving patients with two values, which is to create a score for each patient: If the patient goes from no to yes (that is worsening) the score will be -1, when he/she remains static the score will be 0 when going from yes to no (that is improvement) the score will be 1. Then I will compare the average of the scores between the two groups

What are the advantages and disadvantages of each of these methods? What could be their biases?

Answer 1

You would best respect the pairing of the data, with something similar to McNemar's test. That test, however, is just for things like before/after paired differences and doesn't directly take your two treatments into account.

Your second suggestion, with some modification, is one way to approach your situation. Agresti discusses this in Section 10.2.6 of the second edition of Categorical Data Analysis. As with the McNemar test you only examine those whose status changed. You code those who changed status as 1/0 depending on the direction of change, rather than -1/1 as you propose. You fit

a logistic regression model to those pairs alone, using artificial response $y^* =1$ when $(y_{i1} = 0, y_{i2} = 1)$, $y^* =0$ when $(y_{i1} = 1, y_{i2} = 0)$, no intercept, and predictor values $x^*$.

Here, $y_{i1}$ and $y_{i2}$ are the outcomes (diabetes status in your case) for individual $i$ at times 1 and 2, respectively. A predictor value $x^*$ is the difference in predictor values between the two time points, $x_i^* = x_{i2}-x_{i1}$. In your situation with two treatment groups, you code that with a single binary predictor.

If you had more than 2 time points, you could consider a mixed model instead to deal with both multiple observations within each individual and the missing data. With only 2 time points, however, that doesn't really help. Do think carefully about the nature of your missing data and how best to deal with it.

You also might consider whether an all-or-none diabetes outcome value is wise here. That disease occurs on a spectrum, and modeling of some continuous measure (like fasting blood glucose or hemoglobin A1C) could be highly preferable.