2
$\begingroup$

I am trying to solve following problem in R.

My Model:

DV ~ IV1 * IV2

Whereas IV1 and IV2 are both categorical with many levels (IV1 in the thousands, IV2 around 10) and the DV is binary (1,0 - success or non-success)

What tests should I perform to see if the interaction between IV1 and IV2 (jointly, not on a per-level basis) is statistically significant or not? Is there a statistical measurement that gives me an answer? Is there similar/better model to test for this joint significance?

Thank you for your help!

$\endgroup$

1 Answer 1

3
$\begingroup$

This is a chunk test.

It can be accomplished by testing nested logistic regressions (or whatever models you use, maybe probit). Your "null" model is the model that has both categories but no interaction. Then your "full" model includes those terms but also the interactions terms. The "chunk test" then tests if all of the coefficients on the interaction terms are zero.

I would do this with a likelihood ratio test, and there is fairly simple R code for doing it.

library(lmtest)
set.seed(2023)
N <- 100
y <- rbinom(N, 1, 0.5)
x1 <- as.factor(sample(seq(1, 4), N, replace = T)) # Factor 1 with 4 levels
x2 <- as.factor(sample(seq(1, 3), N, replace = T)) # Factor 2 with 3 levels
L0 <- glm(y ~ x1 + x2, family = binomial)          # Null model
L1 <- glm(y ~ x1*x2, family = binomial)            # Full model
lmtest::lrtest(L0, L1)                             # Likelihood ratio test
lmtest::waldtest(L0, L1)                           # Wald test

This example has no relationship between y and the categorical features, so the p-values are high, but this would work for your case where you do expect some relationship between y and the features.

I have heard this approach called a "partial F-test" when it has been applied to traditional nested linear models.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.