Comparing two logistic regressions

Question

I am working on creating a predictive model using logistic regressions. I am hoping to compare two different populations, using the same set of variables but different data sets with different sample sizes. Is there a good way to compare the two regressions in R to see if they are statistically different from each other?

Edit for clarification: My goal is to determine if the variables affect the outcome differently to a statistical degree. My analysis is about political socialization of two different populations, so I want to try to determine if xx population is socialized differently than yy population to a statistically significant degree. The two logistic models look different, but I am trying to give a more analytical answer than just "they look different".

Answer 1

This sounds like a job for an interaction term and a chunk test.

For two variables $x_1$ and $x_2$ in each regression, the math is as follows; extending to more than just two variables is straightforward. Let $g(p) = \log\left(\dfrac{p}{1 - p}\right)$. Let $x_3$ be an indicator variable that takes $0$ for one group and $1$ for the other.

$$ g\left(\mathbb E\left[y_i\right]\right) = \beta_0 + \beta_1x_{i1} + \beta_2x_{i2} + \beta_3x_{i3} + \beta_4x_{i1} x_{i3} + \beta_5x_{i2} x_{i3} $$

The model includes the original two features, an indicator variable for the group, and interactions between the group indicator and the original two variables.

Nested within such a model, by setting $\beta_3=\beta_4=\beta_5=0$, is the following model that you would have used for each group separately.

$$ g\left(\mathbb E\left[y_i\right]\right) = \beta_0 + \beta_1x_{i1} + \beta_2x_{i2} + $$

You can fit the above model to each group, or you can fit the first model to both groups simultaneously. Then the $x_3$ variable acts like an on/off switch, showing how the regression parameters differ between the two groups.

To test if the groups differ in slope, test $\beta_3=0$. To test if the groups differ in $x_1$, test $\beta_4=0$. To test if the groups differ in $x_2$, test $\beta_5=0$. You also can test $\beta_3=\beta_4=\beta_5=0$ to see if the groups differ in their regressions at all. This testing is no different from any other testing of logistic regressions.

Below, I demonstrate how to do this in R. The final line of lmtest::lrtest uses a likelihood ratio test of nested models to calculate a p-value, either for one coefficient (giving the difference between the parameters for each group regressed separately) or for the three coefficients involving $x_3$ (testing if group membership affects the regression, what I have learned to call a chunk test).

library(lmtest)
set.seed(2023)
N <- 1000
x1 <- runif(N, 0, 1)
x2 <- runif(N, 0, 1)
x3 <- rbinom(N, 1, 0.5)
z <- x1 - x2 + x3 - x1*x3 + x2*x3
p <- 1/(1 + exp(-z))
y <- rbinom(N, 1, p)
L_full <- glm(y ~ x1 + x2 + x3 + x1:x3 + x2:x3, family = binomial)
L0 <- glm(y ~ x1 + x2, family = binomial)
L4 <- glm(y ~ x1 + x2 + x3 + x1:x3)
L5 <- glm(y ~ x1 + x2 + x3 + x2:x3)
L3 <- glm(y ~ x1 + x2 + x3)
lmtest::lrtest(L0, L_full) # I get p = 0.00181 for the full chunk test
lmtest::lrtest(L0, L4) # I get p = 0.03512 for testing beta 4
lmtest::lrtest(L0, L5) # I get p = 0.9272 for testing beta 5
lmtest::lrtest(L0, L3) # I get p = 0.5902 for testing beta 3