0
$\begingroup$

I'm running a logistic regression model with several variables. I'm picking variables with a p less or equal to 0.2 to include in the multivariate model, and a lot of variables have really low p values but there is one categorical variable in which a few of the levels have moderate correlations (p values around 0.1-0.2) so nothing exciting but when I include it in my model, which I have to because I specify the correlation as minimum 0.2, the entire thing breaks apart and nothing is interesting anymore. The p values all becomes mediocre and not significant for the other variables, which were very exciting both on their own and in a multivariate model without that one ruining variable.

Why is this happening?

$\endgroup$
2
  • $\begingroup$ It's not easy to tell for sure with the information supplied but it may be multicollinearity. $\endgroup$
    – Glen_b
    Aug 6 '19 at 16:11
  • $\begingroup$ Can you tell us what your outcome is, what the categorical is and what the previously significantly associated regressors are? What you describe happens very often in regression models: A variable that is marginally associated with an outcome becomes independent when you add another control. This is an interesting finding that typically makes a lot of sense relative to domain knowledge. Here's one reason this happens: The categorical is a common cause of both the outcome and these other predictors. Another: The categorical is a superior measure of something other predictors also measure. $\endgroup$
    – CloseToC
    Aug 6 '19 at 19:15
5
$\begingroup$

You wrote

I'm picking variables with a p less or equal to 0.2 to include in the multivariate model,

This is known as bivariate screening and it is a terrible method of building a model. This has been discussed here before or see Frank Harrell's book Regression Modelling Strategies.

but when I include it in my model, which I have to because I specify the correlation as minimum 0.2

Correlation or p value? In either case, you don't "have to".

The entire thing breaks apart and nothing is interesting anymore. The p values all becomes mediocre and not significant for the other variables, which were very exciting both on their own and in a multivariate model without that one ruining variable.

When a new variable has big effects on the other parameters, that is usually interesting. That is one reason we add control variables. "Ruining" is entirely the wrong word here. I would look into mediation models.

Why is this happening?

Without context it's hard to say more than "because your IVs are not orthogonal to each other".

$\endgroup$
8
  • $\begingroup$ Well what exactly is a good method of building a model then? I've been told this is a great method and that stepwise selction is terrible. $\endgroup$
    – Paze
    Aug 6 '19 at 17:06
  • 1
    $\begingroup$ Well, what you were told is half-right at any rate! $\endgroup$
    – jbowman
    Aug 6 '19 at 17:33
  • $\begingroup$ Then what IS a good method for building a multivariate logistic regression model? I just get more confused the more I try to get the answer to this question. $\endgroup$
    – Paze
    Aug 6 '19 at 18:08
  • $\begingroup$ @Paze You probably mean a multiple logistic regression model, which is one that has several explanatory variables and one response variable. It's time to consult a good textbook--Hosmer and Lemeshow, or maybe Agresti--to get a principled start. $\endgroup$
    – whuber
    Aug 6 '19 at 20:30
  • $\begingroup$ Yes that's what I mean. I'm a medical student and unfortunately cannot learn statistics entirely, but I do have a statistician at hand that helped me set up the project. Unfortunately he is not directly associated with the project and can not be on call for all questions hence I'm here. Is my model still built badly if it is a multiple regression? $\endgroup$
    – Paze
    Aug 7 '19 at 6:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.