1
$\begingroup$

I am trying to model the effect of an event on the odds ratio using logistic regression. I have multiple predictors (binary, categorical and numerical), control variables and a binary outcome. I have 4 months before and 4 months after an event. The data is pooled cross sectional, so I have one observation for each individual but across different times of collection. However, there is no controlled sample, all the sample in the latter 4 months is affected by the event. So I can NOT create a 2x2 matrix where there is controlled versus treated and before versus after the event. I only have 1x2 dimension: before and after the event.

I am trying to compare the coefficients of the predictors of the two logistic regression model, model 1 includes data before the event and model 2 includes data after the event.

I used moderation (interaction) so far but my boss thinks it is not the right way. In this, I put all the data in 1 model and I included an interaction between all the predictors and the event. However, he thinks that this model is daunting because it includes a lot of independent variables (the predictors, a dummy variable for the event, an interaction between each of the predictors and the dummy event and finally the controls).

My objective: To measure the effect of the event on the influence of the predictors (which are binary, categorical and numerical variables) on the binary outcome variable.

I need to find a statistic similar to the t-test: the t-test compares mean values, however I need to compare the beta (the regression coefficients) in this case. Is there any statistic parallel to the t-test that compares the coefficients of 2 regression models?

Thank you

$\endgroup$
2
  • $\begingroup$ How many data do you have? How many of those are 'successes'? How many variables do you have? Do you really need / want to assess whether the event moderates every variable? $\endgroup$ Jul 7, 2021 at 16:08
  • $\begingroup$ @gung-ReinstateMonica The success is around 60%. I am assessing the predictors but not the controls so I need to compare 4 predictors. The data is more than 10,000 observations. $\endgroup$ Jul 8, 2021 at 12:56

2 Answers 2

1
$\begingroup$

Your original idea was right, and your boss is overconcerned about a non-issue. It is very intuitive to try to compare fitted coefficients from stratified models, but that is a dominated strategy. By far, the best way is to include all data in a single model and test interactions.

You state that your boss is concerned because the model "includes a lot of independent variables". This is silly. A basic rule of thumb states that you need at least 15 of the less commonly occurring outcome per variable in a logistic regression model. You state that you have 10k data with 60% success. That means you have 4k failures. That's enough for 266 variables, according to the stated rule of thumb. You have 4. Including interactions will double that. You're fine. Go ahead with your original plan.

$\endgroup$
3
  • $\begingroup$ Thank you so much for the support and clarifications. $\endgroup$ Jul 9, 2021 at 10:51
  • $\begingroup$ You're welcome, @ZeinabElashidy. $\endgroup$ Jul 9, 2021 at 11:30
  • $\begingroup$ If you think one of these answers has resolved your question, you can accept it by clicking the check mark below its vote total to its left. $\endgroup$ Jul 9, 2021 at 11:31
0
$\begingroup$

Your approach seems reasonable, but it can be underpowered given a small sample size and the usage of degrees of freedom due to the interactions effects you added to the regression.

My approach would be similar, but instead of modeling all the variables interacted with the event in a single model, I would have one regression per variable of interest, while controlling for all other variables. This way I would report the statistics of the interaction parameter and the effect sizes of pre and post-event of that single variable while controlling for everything else.

Another approach, if your design matrix is very wide, would be a grouped lasso regression where the group subjected to the regularization would only be the variables of interest interacted with the event variable.

A third approach would be to construct a set of reasonable decision trees and manually checking for when the event variable interacted with a variable of interest leading to a significant final node.

$\endgroup$
1
  • 1
    $\begingroup$ Thank so much for your answer. I think I should run your first approach (modelling each interaction separately) and see the results. $\endgroup$ Jul 8, 2021 at 13:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.