One variable "ruining" an otherwise interesting model

I'm running a logistic regression model with several variables. I'm picking variables with a p less or equal to 0.2 to include in the multivariate model, and a lot of variables have really low p values but there is one categorical variable in which a few of the levels have moderate correlations (p values around 0.1-0.2) so nothing exciting but when I include it in my model, which I have to because I specify the correlation as minimum 0.2, the entire thing breaks apart and nothing is interesting anymore. The p values all becomes mediocre and not significant for the other variables, which were very exciting both on their own and in a multivariate model without that one ruining variable.

Why is this happening?

• It's not easy to tell for sure with the information supplied but it may be multicollinearity. Aug 6, 2019 at 16:11
• Can you tell us what your outcome is, what the categorical is and what the previously significantly associated regressors are? What you describe happens very often in regression models: A variable that is marginally associated with an outcome becomes independent when you add another control. This is an interesting finding that typically makes a lot of sense relative to domain knowledge. Here's one reason this happens: The categorical is a common cause of both the outcome and these other predictors. Another: The categorical is a superior measure of something other predictors also measure. Aug 6, 2019 at 19:15

You wrote

I'm picking variables with a p less or equal to 0.2 to include in the multivariate model,

This is known as bivariate screening and it is a terrible method of building a model. This has been discussed here before or see Frank Harrell's book Regression Modelling Strategies.

but when I include it in my model, which I have to because I specify the correlation as minimum 0.2

Correlation or p value? In either case, you don't "have to".

The entire thing breaks apart and nothing is interesting anymore. The p values all becomes mediocre and not significant for the other variables, which were very exciting both on their own and in a multivariate model without that one ruining variable.

When a new variable has big effects on the other parameters, that is usually interesting. That is one reason we add control variables. "Ruining" is entirely the wrong word here. I would look into mediation models.

Why is this happening?

Without context it's hard to say more than "because your IVs are not orthogonal to each other".

• Well what exactly is a good method of building a model then? I've been told this is a great method and that stepwise selction is terrible.
– Paze
Aug 6, 2019 at 17:06
• Well, what you were told is half-right at any rate! Aug 6, 2019 at 17:33
• Then what IS a good method for building a multivariate logistic regression model? I just get more confused the more I try to get the answer to this question.
– Paze
Aug 6, 2019 at 18:08
• @Paze You probably mean a multiple logistic regression model, which is one that has several explanatory variables and one response variable. It's time to consult a good textbook--Hosmer and Lemeshow, or maybe Agresti--to get a principled start.
– whuber
Aug 6, 2019 at 20:30
• Yes that's what I mean. I'm a medical student and unfortunately cannot learn statistics entirely, but I do have a statistician at hand that helped me set up the project. Unfortunately he is not directly associated with the project and can not be on call for all questions hence I'm here. Is my model still built badly if it is a multiple regression?
– Paze
Aug 7, 2019 at 6:55