71
$\begingroup$

The coefficient of an explanatory variable in a multiple regression tells us the relationship of that explanatory variable with the dependent variable. All this, while 'controlling' for the other explanatory variables.

How I have viewed it so far:

While each coefficient is being calculated, the other variables are not taken into account, so I consider them to be ignored.

So am I right when I think that the terms 'controlled' and 'ignored' can be used interchangeably?

$\endgroup$
4
  • 2
    $\begingroup$ I wasn't so enthused about this question until I saw the two figured you inspired @gung to offer. $\endgroup$
    – DWin
    Dec 7, 2013 at 4:22
  • 1
    $\begingroup$ You weren't aware of the conversation we were having elsewhere that motivated this question, @DWin. It was too much to try to explain this in a comment, so I asked the OP to make it a formal question. I actually think explicitly bringing out the distinction b/t ignoring & controlling for other variables in regression is a great question, & I glad it got discussed here. $\endgroup$ Dec 7, 2013 at 4:27
  • 2
    $\begingroup$ see also the first diagram here $\endgroup$
    – Glen_b
    Dec 7, 2013 at 9:30
  • 1
    $\begingroup$ Is the data used in this question available so we could run it ourselves as a educating sample. $\endgroup$
    – Larry
    Aug 23, 2017 at 3:56

2 Answers 2

114
$\begingroup$

Controlling for something and ignoring something are not the same thing. Let's consider a universe in which only 3 variables exist: $Y$, $X_1$, and $X_2$. We want to build a regression model that predicts $Y$, and we are especially interested in its relationship with $X_1$. There are two basic possibilities.

  1. We could assess the relationship between $X_1$ and $Y$ while controlling for $X_2$:
    $$ Y = \beta_0 + \beta_1X_1 + \beta_2X_2 $$ or,
  2. we could assess the relationship between $X_1$ and $Y$ while ignoring $X_2$:

    $$ Y = \beta_0 + \beta_1X_1 $$

Granted, these are very simple models, but they constitute different ways of looking at how the relationship between $X_1$ and $Y$ manifests. Often, the estimated $\hat\beta_1$s might be similar in both models, but they can be quite different. What is most important in determining how different they are is the relationship (or lack thereof) between $X_1$ and $X_2$. Consider this figure:

enter image description here

In this scenario, $X_1$ is correlated with $X_2$. Since the plot is two-dimensional, it sort of ignores $X_2$ (perhaps ironically), so I have indicated the values of $X_2$ for each point with distinct symbols and colors (the pseudo-3D plot below provides another way to try to display the structure of the data). If we fit a regression model that ignored $X_2$, we would get the solid black regression line. If we fit a model that controlled for $X_2$, we would get a regression plane, which is again hard to plot, so I have plotted three slices through that plane where $X_2=1$, $X_2=2$, and $X_2=3$. Thus, we have the lines that show the relationship between $X_1$ and $Y$ that hold when we control for $X_2$. Of note, we see that controlling for $X_2$ does not yield a single line, but a set of lines.

enter image description here

Another way to think about the distinction between ignoring and controlling for another variable, is to consider the distinction between a marginal distribution and a conditional distribution. Consider this figure:

enter image description here

(This is taken from my answer here: What is the intuition behind conditional Gaussian distributions?)

If you look at the normal curve drawn to the left of the main figure, that is the marginal distribution of $Y$. It is the distribution of $Y$ if we ignore its relationship with $X$. Within the main figure, there are two normal curves representing conditional distributions of $Y$ when $X_1 = 25$ and $X_1 = 45$. The conditional distributions control for the level of $X_1$, whereas the marginal distribution ignores it.

$\endgroup$
19
  • 2
    $\begingroup$ Gung, this is enlightening, I am glad I made the mistake of using the word 'ignore' in my answer to that question. Im now going to try find out how exactly statistical packages 'control' for the other variables. (My first thought is they use some measure like the pearson correlation coefficient. With many explanatory variables, things would get messy though) Thank you for this answer! $\endgroup$ Dec 7, 2013 at 2:50
  • 1
    $\begingroup$ You're welcome, @garciaj, although I'm not done yet ;-). I'm looking for another figure; I may have to make it from scratch. $\endgroup$ Dec 7, 2013 at 2:52
  • 4
    $\begingroup$ The crucial idea in the first figure is that those points lie in a three-dimensional space, w/ the red circles on a flat plane at the computer screen, the blue triangles on a parallel plane a little in front of the screen & the green pluses on a plane a little in front of that. The regression plane tilts downward to the right, but slopes upward as it moves out from the screen towards you. Note that this phenomenon occurs because X1 & X2 are correlated, if they were uncorrelated, the estimated betas would be the same. $\endgroup$ Dec 7, 2013 at 14:52
  • 1
    $\begingroup$ And this kind of correlation among predictors (e.g., @gung scenario) is what usually underlies a case of the Simpson's paradox. In a universe with more than three variables, it is wise to remember that it may be lurking your inferences (d'oh!). $\endgroup$
    – FairMiles
    Apr 19, 2016 at 15:03
  • 2
    $\begingroup$ @MSIS, when you control for a variable in a model, the model tries to hold it constant (fixed) for the sake of estimating everything else in the model. However, this is just an attempt & subject to random error, so it isn't necessarily identical to what you would get if you ran a study w/ a variable physically fixed at a given value. $\endgroup$ Aug 21, 2016 at 14:18
10
$\begingroup$

They are not ignored. If they were 'ignored' they would not be in the model. The estimate of the explanatory variable of interest is conditional on the other variables. The estimate is formed "in the context of" or "allowing for the impact of" the other variables in the model.

$\endgroup$
2
  • $\begingroup$ The estimate is of course subject to other variables. But we must purify it by introducing the so-called other factors in the model. However, sometimes these factors may be of categorical nature and cause more problems than give a valid solution. $\endgroup$
    – user10619
    Dec 25, 2013 at 10:08
  • $\begingroup$ That's certainly true and it doesn't arise just for categorical variables. Continuous and ordinal variables can cause invalid inferences if the underlying science and reality are not taken into account properly, $\endgroup$
    – DWin
    Mar 3, 2021 at 23:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.