# How to test if those 2 regression models are fundamentally equivalent?

I want to test/prove if two regressions are fundamentally the same thing.

But first, make this a reproducible example.

Create the levels for each factor variable

levels_a <- c("A1", "A2", "A3", "A4")
levels_b <- c("B1", "B2", "B3")
levels_c <- c("C1", "C2")


Generate all possible combinations of factor levels

combinations <- expand.grid(levels_a, levels_b, levels_c)


Create a data frame with the factor variables and the dependent variable

data <- data.frame(
a = factor(combinations$Var1), b = factor(combinations$Var2),
c = factor(combinations$Var3), DepVar = sample(c(0, 1), size = nrow(combinations), replace = TRUE) )  My original regression is original <- glm(formula = DepVar ~ a*b*c , family = "binomial", data = data) summary(original)  A, B and C are all factors. A has four levels, b has three levels, and c has 2 levels. Therefore, the "original" regression has an intercept and 3 terms from a, 2 terms from b, 1 term from c, then 6 (3*2) from a and b, and then, lastly, 11 terms from c (c= 1 times all before but itself). In sum, these are 23 terms plus intercept. However, I want to use the sjplot for interactions, and this packet only allows one interaction. I want to plot the c- to the rest- interaction. So I thought maybe I could do it like this: My second regression is data$interaction_var <- interaction(data$a, data$b)

test <- glm(formula = DepVar ~ interaction_var * c , family = "binomial",
data = data)
summary(test)


While this interaction can be plotted with "sj", I want to find out if this regression always produces the same predicted values as the first regression, as this regression has different terms:

Again, it has an intercept, all 12 interactions (minus 1) from a and b, 1 from c, and again 11 (c=1 times everything so far but itself). So, it also has 23 components.

How can I show that they are identical in terms of predictions?

How can I show that they are identical in terms of predictions?

If you just want a computational demonstration (rather than a mathematical proof),

all.equal(predict(original), predict(test))


should do it ... (all.equal() automatically uses an "equivalent within numerical tolerances" test, rather than testing that the two predictions are identical up to the full floating-point precision).

You did say

always produces the same predicted values

[emphasis added] (not "produces the same predicted values for the original data set"), but something pretty weird would have to be happening (or you'd have to pick a pathological data set) in order for the predictions to be identical on the training set but different for some other set of input.

I suppose that the models will be identical and produce equal predicted values if they have same coefficients:

epsylon <- 10^-6 # set maximum allowed difference between coefficients
length(coefficients(original)) == length(coefficients(test)) &&
all(abs(coefficients(original) - coefficients(test)) < epsylon
)

[1] FALSE


And we can print different coefficients in a human readible form:

data.frame(coefficients(original),
coefficients(test))[
abs(coefficients(original) - coefficients(test)) >= epsylon,
]

coefficients.original. coefficients.test.
bB3                  -5.113214e+01       3.246735e-09
cC2                   1.014892e-07      -5.113214e+01
aA2:bB2               5.113214e+01       1.029394e-07
aA3:bB2              -4.707198e-09      -5.113214e+01
aA4:bB2              -1.057436e-07      -5.113214e+01
aA2:bB3               5.113214e+01       5.457133e-09
aA3:bB3               1.022643e+02       5.717070e-10
aA4:bB3               5.113214e+01      -1.227631e-09
bB3:cC2               5.113214e+01      -5.113214e+01
aA2:bB2:cC2          -5.113214e+01       1.227629e-09
aA3:bB2:cC2           5.113214e+01      -9.859460e-08
aA2:bB3:cC2          -1.022643e+02       1.227591e-09
aA3:bB3:cC2          -5.113214e+01      -2.211011e-06
aA4:bB3:cC2          -1.022643e+02      -5.113214e+01

• Comparing the coefficients is relevant only when the variables are coded identically in both models. Why not just compare the predictions to each other?
– whuber
Oct 10, 2023 at 21:02
• I think all the variables a, b, and c are the same, so they are also "coded identically". I use the same variables from the same dataset. Oct 11, 2023 at 10:11