Non-significant predictor variable becomes significant when two control variables are introduced into model

Question

I want to investigate the correlation of $A$ and $B$ on $X$, so i do a multiple linear Regression. On their own $A$ and $B$ are significant predictors of $X$ in a simple linear regression.

When i test a multiple linear regression with with $A$ and $B$ as Predictors, only $B$ is a significant Predictor. I thought this is because they $A$ and $B$ are redundandt. However, when i do a multiple regression insert a control variable (i.e. Gender and Age), both $A$ and $B$ become significant. One of the CVs is not enough for this.

Could this be a problem related to overfitting?

Answer 1

This might not be a "problem" at all.

First of all, it's hard to know what you want to do here without a clearly articulated research question - WHY do you think A and B are associated with "X" (the dependent variable...which we usually refer to as "Y"...which is what I will do for the rest of this answer). There is no such thing as a "good" or "bad" model in the abstract, only models that are useful or not useful for answering a particular research question. One reason I worry you haven't done that is that you included both A and B at the same time means that you are looking for the "effect" of A on Y holding B constant....and the effect of B holding A constant. If that's really what you want to know, that's fine, but you should make sure it really is.

Second, it's often the case that the significance of variable changes when you add controls to a model. That's how controls work. It's your job as a researcher to figure out what these changes means. What these models are telling you is that there is no significant association between your predictor A and Y when you don't control for age and gender, but there is a significant association when you do. That might just be how reality works.

For example, let's say you are testing whether enrolling in class (variable "A") leads to better scores on some sort of test. Let's say that the class itself is worthless, but that older females are both more likely to enroll in the class but less likely to do well on the test. If that were true then you might not see a significant effect effect for variable B (taking the class) in a model that doesn't control for gender and age, or only one of them, but you do see a significant effect when you control for both. Also, let's assume that variable A is associated with test score as well, but isn't associated with age or gender. That set of relationships would produce exactly the results you describe. It's not a problem, just a story about how things are related to each other. Does that help you answer your research question or not?

This is why it's good to clearly lay out the casual relationships you think might exist before running your model. Structural equation diagrams ("boxes and arrows") are especially helpful with this.