1
$\begingroup$

I run a multiple regression with an IV that could not explain any variance in the DV. Is it possible that if I add another variable in the model as a control that the IV entered first would start to explain variance in the DV?

$\endgroup$
1

1 Answer 1

1
$\begingroup$

I actually think it is:

Imagine you have a response variable $Y = Y_1, Y_2, \dots Y_n \sim N(0,1)$ and an initial control variable $X_1 = X_{11}, X_{12} \dots X_{1n} \sim N(0,1)$. Then it is clear that $X_1$ doesn't explain Y. However imagine you then have $X_2 = Y_1 - X_{11}, Y_2- X_{12}, \dots Y_n - X_{1n}$ then you have the simple relation: \begin{equation}Y = X_1 + X_2 \end{equation} and thus the previously useless $X_1$ is now part of a model that has 100% accuracy.

i.e.

>>> from sklearn.linear_model import LinearRegression
>>> import numpy as np
>>> Y = np.random.normal(size=1000)
>>> X = np.random.normal(size=1000)
>>> X2 = Y - X

>>> lr = LinearRegression()
>>> lr.fit(np.array(X).reshape(1000,1), Y)
LinearRegression(copy_X=True, fit_intercept=True, n_jobs=1, normalize=False)
>>> lr.score(np.array(X).reshape(1000,1), Y)
0.0011271850072254175


>>> lr = LinearRegression()
lr.fit(np.array(X2).reshape(1000,1), Y)
LinearRegression(copy_X=True, fit_intercept=True, n_jobs=1, normalize=False)
>>> lr.score(np.array(X2).reshape(1000,1), Y)
0.4728675730093943



>>> lr = LinearRegression()
>>> lr.fit(np.array([X, X2]).T, Y)
LinearRegression(copy_X=True, fit_intercept=True, n_jobs=1, normalize=False)
>>> lr.score(np.array([X, X2]).T, Y)
1.0

So perfect prediction in a combined linear model despite no correlation for $X_1$ when fit individually.

Is this what you meant?

$\endgroup$
2
  • $\begingroup$ Thank you for your elaborate answer. I was thinking about a situation where a DV is affected by many factors and because of this there is so much "noise" that the "noise" might be the reason why there is no correlation between X1 and Y in a certain sample. So if I would control some of those factors that cause noise, could that increase the correlation between X1 and Y? $\endgroup$
    – Marsupaani
    May 6, 2018 at 13:40
  • $\begingroup$ If the noise has a large magnitude in comparison to your desired effect and you are able to model it accurately then yes, this is certainly possible $\endgroup$ May 6, 2018 at 13:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.