# Multiple linear regression: Do all independent variables need to have good adjusted R-squared independently?

I'm very sorry if this should be obvious, I'm just feeling a little lost with this assignment..

I have four independent variables X1,X2,X3,X4 plus a constant, modelled against Y. I know X4 to be heavily correlated already, it's mostly a control variable. I've checked for multicollinearity. There are 52 observations. These are the results with only one independent variable at a time:

X1 X2 X3 X4
coefficient 0.77 -0.32 0.34 0.95
p-value 0.0001 0.03 0.027 0.00005
adjusted r-squared 0.567 0.0632 0.074 0.645

And these are the results when combined with X4:

(X1, X4) (X2, X4) (X3, X4)
coefficient (0.43, 0.64) (-0.34, 0.96) (-0.03, 0.93)
p-value (0.00001, 0.0000000084) (0.000031, 0.00000001) (0.73, 0.000006)

I'm not sure if it's relevant, but the constant terms is varying positive and negative, sometimes with a significant p-value and sometimes not.

My question is: X2 only has 0.06 adjusted r-squared with Y by itself, while X4 has 0.645 by itself. But combined, r-squared increases to 0.747. Does that mean something is wrong with my model? Or that the tiny variance in Y that X2 explains (6%) is not included in X4, so that X2 is actually a significant variable in the model? Is 0.1 increase in r-squared even enough to say the combined model is better? Please help!

• Welcome to Cross Validated! What is the purpose of this assignment? Usually we fit one model with all relevant predictors: Y ~ X1 + X2 + X3 + X4 (model with main effects for all predictors without interactions). Why are you fitting multiple regressions and what do you hope to learn from those? May 8, 2022 at 12:37
• Thank you!! The purpose is to determine whether any of the independent variables X1,X2,X3 are correlated to Y. None of them have to be, I picked the variables myself. Y = how many people shop online, X4 = how many people have access to internet, X1,X2,X3 other markofactors that could possible make people shop more/less. Should I be fitting a model with all four? How will I tell which are individually important? I should mention the variables have different ranges but none is more than lightly skewed. May 9, 2022 at 16:03
• So here is my confusion: You say your purpose is to "determine whether any of the independent variables X1,X2,X3 are correlated to Y". But the rest of your comment make it sound that your purpose is to model Y (to predict future Y? to estimate the relationships between X and Y?). May 9, 2022 at 16:12
• Estimating pairwise correlations and modeling an outcome Y given features X are two very different goals. And steps you would take for one goal don't necessarily make sense for the other. May 9, 2022 at 16:12
• @dipetkov Yes so you can see I haven't really got a grasp on this... I'm really sorry for the confusion. If inference is my goal, e.g. conclusions like "X1 and X2 are correlated to Y, X2 more so than X1", how would you suggest I model them and which metrics do I look at? Sorry for the broad question May 11, 2022 at 18:50

Given that the coefficient of $$X_2$$ doesn't change (-0.32 vs -0.34) the main responsible for this behaviour is $$Var(B_2)$$. It seems to go down very quickly, lowering the p-value even more. In an "ideal" model the p-value of $$X_2$$ after the addition of a new variable should be less significant or stay the same. To answer your questions:
1. "Does that mean something is wrong with my model?" Not quite. Given that $$X_2$$ was already significant, it's not a great deal. However, it depends on the results of the full model, with all the $$X$$s, which is not shown.
2. "Or that the tiny variance in Y that X2 explains (6%) is not included in X4, so that X2 is actually a significant variable in the model?" Theoretically yes, but the numbers don't add up exactly, so numerically is not correct. One cause is the alleged multicollinearity between $$X_2$$ and $$X_4$$.
3. "Is 0.1 increase in r-squared even enough to say the combined model is better?" --> Yes, but don't forget the fact that $$X_2$$ was already significant in the bivariate model ($$Y$$ against $$X_2$$). If it weren't, then it would be a false positive.