Skip to main content
edited body
Source Link
Muno
  • 415
  • 2
  • 14

Suppose that we want to predict $y$ using a subset of the variables $x_1 \dots x_n$ using linear regression. Suppose I regress $y$ on $x_1$, from which I obtain fit $f_1$, regress $y$ on $x_2$, from which I obtain $f_2$ and then regress $y$ on $x_3$, from which I obtain $f_3$.

Facts:

The slope of $f_1$ is almost $0$ (that is, the best fit line is almost constant). The slope of $f_3$ is greater in magnitude than the slope of $f_2$.

Questions:

  1. Can I claim that $x_1$ is not an important predictor?

  2. Can I claim that $x_3$ is a more important predictor than $f_2$?

I believe the answer to 2) is "no." Simply fabricate a relationship $y \sim 0.00001(x_1)$, create a variable $x_1 \sim N(0,10)$$x_2 \sim N(0,10)$ and then $y \sim x_2$ will have a greater slope than $y \sim x_1$, even though $x_2$ is clearly unimportant.


Concrete Examples

Here are some concrete examples of an output variable (yield_rainfed_ana) regressed on three different, input variables. The best fit lines associated with these 3, following examples mimic, respectively, the lines $f_1,f_2,f_3$ that I described earlier.


enter image description here

enter image description here

enter image description here

Suppose that we want to predict $y$ using a subset of the variables $x_1 \dots x_n$ using linear regression. Suppose I regress $y$ on $x_1$, from which I obtain fit $f_1$, regress $y$ on $x_2$, from which I obtain $f_2$ and then regress $y$ on $x_3$, from which I obtain $f_3$.

Facts:

The slope of $f_1$ is almost $0$ (that is, the best fit line is almost constant). The slope of $f_3$ is greater in magnitude than the slope of $f_2$.

Questions:

  1. Can I claim that $x_1$ is not an important predictor?

  2. Can I claim that $x_3$ is a more important predictor than $f_2$?

I believe the answer to 2) is "no." Simply fabricate a relationship $y \sim 0.00001(x_1)$, create a variable $x_1 \sim N(0,10)$ and then $y \sim x_2$ will have a greater slope than $y \sim x_1$, even though $x_2$ is clearly unimportant.


Concrete Examples

Here are some concrete examples of an output variable (yield_rainfed_ana) regressed on three different, input variables. The best fit lines associated with these 3, following examples mimic, respectively, the lines $f_1,f_2,f_3$ that I described earlier.


enter image description here

enter image description here

enter image description here

Suppose that we want to predict $y$ using a subset of the variables $x_1 \dots x_n$ using linear regression. Suppose I regress $y$ on $x_1$, from which I obtain fit $f_1$, regress $y$ on $x_2$, from which I obtain $f_2$ and then regress $y$ on $x_3$, from which I obtain $f_3$.

Facts:

The slope of $f_1$ is almost $0$ (that is, the best fit line is almost constant). The slope of $f_3$ is greater in magnitude than the slope of $f_2$.

Questions:

  1. Can I claim that $x_1$ is not an important predictor?

  2. Can I claim that $x_3$ is a more important predictor than $f_2$?

I believe the answer to 2) is "no." Simply fabricate a relationship $y \sim 0.00001(x_1)$, create a variable $x_2 \sim N(0,10)$ and then $y \sim x_2$ will have a greater slope than $y \sim x_1$, even though $x_2$ is clearly unimportant.


Concrete Examples

Here are some concrete examples of an output variable (yield_rainfed_ana) regressed on three different, input variables. The best fit lines associated with these 3, following examples mimic, respectively, the lines $f_1,f_2,f_3$ that I described earlier.


enter image description here

enter image description here

enter image description here

Source Link
Muno
  • 415
  • 2
  • 14

Are scatter plots between independent and dependent variables reliable ways of refuting linear relationships?

Suppose that we want to predict $y$ using a subset of the variables $x_1 \dots x_n$ using linear regression. Suppose I regress $y$ on $x_1$, from which I obtain fit $f_1$, regress $y$ on $x_2$, from which I obtain $f_2$ and then regress $y$ on $x_3$, from which I obtain $f_3$.

Facts:

The slope of $f_1$ is almost $0$ (that is, the best fit line is almost constant). The slope of $f_3$ is greater in magnitude than the slope of $f_2$.

Questions:

  1. Can I claim that $x_1$ is not an important predictor?

  2. Can I claim that $x_3$ is a more important predictor than $f_2$?

I believe the answer to 2) is "no." Simply fabricate a relationship $y \sim 0.00001(x_1)$, create a variable $x_1 \sim N(0,10)$ and then $y \sim x_2$ will have a greater slope than $y \sim x_1$, even though $x_2$ is clearly unimportant.


Concrete Examples

Here are some concrete examples of an output variable (yield_rainfed_ana) regressed on three different, input variables. The best fit lines associated with these 3, following examples mimic, respectively, the lines $f_1,f_2,f_3$ that I described earlier.


enter image description here

enter image description here

enter image description here