Take the 2-minute tour ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

The coefficient of an explanatory variable in a multiple regression tells us the relationship of that explanatory variable with the dependent variable. All this, while 'controlling' for the other explanatory variables.

How I have viewed it so far:

While each coefficient is being calculated, the other variables are not taken into account, so I consider them to be ignored.

So am I right when I think that the terms 'controlled' and 'ignored' can be used interchangeably?

share|improve this question
    
I wasn't so enthused about this question until I saw the two figured you inspired @gung to offer. –  DWin Dec 7 '13 at 4:22
    
You weren't aware of the conversation we were having elsewhere that motivated this question, @DWin. It was too much to try to explain this in a comment, so I asked the OP to make it a formal question. I actually think explicitly bringing out the distinction b/t ignoring & controlling for other variables in regression is a great question, & I glad it got discussed here. –  gung Dec 7 '13 at 4:27
2  
see also the first diagram here –  Glen_b Dec 7 '13 at 9:30

2 Answers 2

up vote 28 down vote accepted

Controlling for something and ignoring something are not the same thing. Let's consider a universe in which only 3 variables exist: $Y$, $X_1$, and $X_2$. We want to build a regression model that predicts $Y$, and we are especially interested in its relationship with $X_1$. There are two basic possibilities.

  1. We could assess the relationship between $X_1$ and $Y$ while controlling for $X_2$:
    $$ Y = \beta_0 + \beta_1X_1 + \beta_2X_2 $$ or,
  2. we could assess the relationship between $X_1$ and $Y$ while ignoring $X_2$:

    $$ Y = \beta_0 + \beta_1X_1 $$

Granted, these are very simple models, but they constitute different ways of looking at how the relationship between $X_1$ and $Y$ manifests. Often, the estimated $\hat\beta_1$s might be similar in both models, but they can be quite different. What is most important in determining how different they are is the relationship (or lack thereof) between $X_1$ and $X_2$. Consider this figure:

enter image description here

In this scenario, $X_1$ is correlated with $X_2$. Since the plot is two-dimensional, it sort of ignores $X_2$ (perhaps ironically), so I have indicated the values of $X_2$ for each point with distinct symbols and colors (the pseudo-3D plot below provides another way to try to display the structure of the data). If we fit a regression model that ignored $X_2$, we would get the solid black regression line. If we fit a model that controlled for $X_2$, we would get a regression plane, which is again hard to plot, so I have plotted three slices through that plane where $X_2=1$, $X_2=2$, and $X_2=3$. Thus, we have the lines that show the relationship between $X_1$ and $Y$ that hold when we control for $X_2$. Of note, we see that controlling for $X_2$ does not yield a single line, but a set of lines.

enter image description here

Another way to think about the distinction between ignoring and controlling for another variable, is to consider the distinction between a marginal distribution and a conditional distribution. Consider this figure:

enter image description here

(This is taken from my answer here: What is the intuition behind conditional Gaussian distributions?)

If you look at the normal curve drawn to the left of the main figure, that is the marginal distribution of $Y$. It is the distribution of $Y$ if we ignore its relationship with $X$. Within the main figure, there are two normal curves representing conditional distributions of $Y$ when $X_1 = 25$ and $X_1 = 45$. The conditional distributions control for the level of $X_1$, whereas the marginal distribution ignores it.

share|improve this answer
    
Gung, this is enlightening, I am glad I made the mistake of using the word 'ignore' in my answer to that question. Im now going to try find out how exactly statistical packages 'control' for the other variables. (My first thought is they use some measure like the pearson correlation coefficient. With many explanatory variables, things would get messy though) Thank you for this answer! –  garciaj Dec 7 '13 at 2:50
1  
You're welcome, @garciaj, although I'm not done yet ;-). I'm looking for another figure; I may have to make it from scratch. –  gung Dec 7 '13 at 2:52
    
I added the last part, @garciaj, although you may already understand the idea at this point. Regarding how it's done, it simply falls out of the math of finding estimated slopes that minimize the OLS loss function (cf my comment at the linked answer &/or my answer here). For an intuition, you could think of each variable but 1 being held at their means & then the slope of the remaining variable is found (at least when there are no interactions). –  gung Dec 7 '13 at 3:56
    
You illustrate a critical point. The "reversal", i.e. change of sign, of an estimated effect conditional on a regressor is not something that is often handled well, but your new illustration makes it very clear how that result might occur. –  DWin Dec 7 '13 at 4:20
2  
The crucial idea in the first figure is that those points lie in a three-dimensional space, w/ the red circles on a flat plane at the computer screen, the blue triangles on a parallel plane a little in front of the screen & the green pluses on a plane a little in front of that. The regression plane tilts downward to the right, but slopes upward as it moves out from the screen towards you. Note that this phenomenon occurs because X1 & X2 are correlated, if they were uncorrelated, the estimated betas would be the same. –  gung Dec 7 '13 at 14:52

They are not ignored. If they were 'ignored' they would not be in the model. The estimate of the explanatory variable of interest is conditional on the other variables. The estimate is formed "in the context of" or "allowing for the impact of" the other variables in the model.

share|improve this answer
    
The estimate is of course subject to other variables. But we must purify it by introducing the so-called other factors in the model. However, sometimes these factors may be of categorical nature and cause more problems than give a valid solution. –  subhash c. davar Dec 25 '13 at 10:08

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.