# Can an uninformative control variable become useful?

I run a multiple regression with an IV that could not explain any variance in the DV. Is it possible that if I add another variable in the model as a control that the IV entered first would start to explain variance in the DV?

• – mkt
Commented Sep 13, 2018 at 18:18

I actually think it is:

Imagine you have a response variable $Y = Y_1, Y_2, \dots Y_n \sim N(0,1)$ and an initial control variable $X_1 = X_{11}, X_{12} \dots X_{1n} \sim N(0,1)$. Then it is clear that $X_1$ doesn't explain Y. However imagine you then have $X_2 = Y_1 - X_{11}, Y_2- X_{12}, \dots Y_n - X_{1n}$ then you have the simple relation: $$Y = X_1 + X_2$$ and thus the previously useless $X_1$ is now part of a model that has 100% accuracy.

i.e.

>>> from sklearn.linear_model import LinearRegression
>>> import numpy as np
>>> Y = np.random.normal(size=1000)
>>> X = np.random.normal(size=1000)
>>> X2 = Y - X

>>> lr = LinearRegression()
>>> lr.fit(np.array(X).reshape(1000,1), Y)
LinearRegression(copy_X=True, fit_intercept=True, n_jobs=1, normalize=False)
>>> lr.score(np.array(X).reshape(1000,1), Y)
0.0011271850072254175

>>> lr = LinearRegression()
lr.fit(np.array(X2).reshape(1000,1), Y)
LinearRegression(copy_X=True, fit_intercept=True, n_jobs=1, normalize=False)
>>> lr.score(np.array(X2).reshape(1000,1), Y)
0.4728675730093943

>>> lr = LinearRegression()
>>> lr.fit(np.array([X, X2]).T, Y)
LinearRegression(copy_X=True, fit_intercept=True, n_jobs=1, normalize=False)
>>> lr.score(np.array([X, X2]).T, Y)
1.0


So perfect prediction in a combined linear model despite no correlation for $X_1$ when fit individually.

Is this what you meant?

• Thank you for your elaborate answer. I was thinking about a situation where a DV is affected by many factors and because of this there is so much "noise" that the "noise" might be the reason why there is no correlation between X1 and Y in a certain sample. So if I would control some of those factors that cause noise, could that increase the correlation between X1 and Y? Commented May 6, 2018 at 13:40
• If the noise has a large magnitude in comparison to your desired effect and you are able to model it accurately then yes, this is certainly possible Commented May 6, 2018 at 13:44