# Correcting bias in non-equivalent group analysis with logistic regression

I have constructed two groups based on observational data: control (C) and treatment (T). T was exposed to a feature that C was not exposed to. My total number of observations is very high (> 300K).

I want to measure the effect of being in T compared to being in C on a binary outcome O. First I simply calculate the mean of O for each group, which I interpret as probabilities:

P(O=1 | C) = n(O = 1) / n(T=0) = 0.0044
P(O=1 | T) = n(O = 1) / n(T=1) = 0.0061


The issue is that my groups are not equivalent. I cannot apply random sampling to get equivalent groups, so I tried to get groups as close to one another given my constraints but I know they are not equivalent. To see this, I measure O in a previous period, when there is no treatment difference between C and T:

P(Opre = 1 | C) = 0.002
P(Opre = 1 | T) = 0.004


Conducting a chi-square independence test confirms that my two groups are not equivalent.

I want to correct for this bias, and I thought fitting a logistic regression with T and Opre as predictors could be interesting:

O = intercept + theta0 * T + theta1 * Opre + e


I can compare theta0 with the theta0 found by simply fitting:

O = intercept + theta0 * T + e


However, while theta0 in the first model is lower than in the second model, it is only a little bit lower (0.4118 vs 0.4841) and the probability ratio ends up being only slightly lower in the "bias-corrected" model than in the "biased" model, although predicted probabilities for each group are much lower in the "bias-corrected" model. I find this very unrealistic given the strong difference between each group's Opre.

I think this means my method is inefficient. Do I need to add more features to correct bias, or do I need to use a different method entirely? Also, why is this method inefficient?

Note that when I compare predictions, I do this by comparing the predictions for

(T=1, Opre = mean(Opre | T=1)) ; (T=1, Opre = mean(Opre | T=0)) in Model 1


and the predictions for

(T=1) ; (T=0) in Model 2.