What does 'Controlling for' mean in regression? [duplicate]

Question

I am working towards completing my undergrad honours thesis and I am in the process of analyzing and writing up my discussion section that is dealing with some form of multiple regression (can't figure out which it is based on my data and what my professor is looking for - In our meetings she can never articulate what I am looking for properly so I just agree and find my answers elsewhere). I have done stats in the past (Worked in SPSS), had no problem in the class with regression, but we only dealt with regression relationships that had 1 dependent and 1 independent variable (y = a + bX). In my analysis I am running 5 different models, with the first model having 1 dependent, and 2 independent, the next has 1 dependent and 3 independent, etc.

The Issue: In the final coefficient table in SPSS after running a regression analyses, I am having troubles understanding how/ why/ the meaning of the values changing. Please correct me if I am wrong, but when you run a regression analyses with multiple independent variables and 1 dependent variable, your answer will look something like this: Lets say for this example there are 2 Independent variables, variable 1 is significant, and variable 2 is not significant: "Variable1 has an impact on the dependent variable while controlling for variable 2. Variable 2 has no impact on the dependent variable while controlling for variable 1"

I am somewhat certain this statement is correct although rather limited in its meaning in the data. At this point that is ok as I can get assistance making this statement stronger within the thesis. My question then is as follows: What is meant by 'controlling for variable 1/ 2'? When we say controlling for, what is actually going on? How is the data actually controlling for it? The way I'd understand it is from a simple equation like this:

Constant (B = 5), Variable 1 (B = 10, Sig. = 0.04), Variable 2 (B = 15, Sig. = 0.98)

y = 5 + 10(x) + 5 + 15(x). From here, if we were examining variable 1 controlling for variable 2, I am thinking the equation would then turn into: y = 5 + 10(x) + 5 + 15(0) so we'd be left with y = 5 + 10(x) + 15?

Sorry to make this post extremely long but thank you for you in advance for any assistance in the matter.

TLDR: What does 'controlling for' mean in a hierarchical or standard or multivariate regression analysis? Please explain in the simplest terms as I am not a stats major nor will I understand complex responses.

Thanks again,

Michael

Answer 1

I'm not totally sure what your example is communicating, but let me see if I can answer you question without it. "Controlling for" means accounting for any shared explanatory power your two variables may have. Let's imagine we had a dataset on children's heights, weights, and ages. Using common sense, we know that taller people tend to weigh more (on average) than shorter people. And that, for children, height and weight probably tell us something about age. So let's say we want to nail down the relationship between weight and age and formally test whether weight is related to age. You could run a simple regression, like you ran in your stats course, that looks like this:

Model 1: $$age = \beta_0 + \beta_1weight + \epsilon$$

Where the beta's are the slopes and intercepts, age represents the age values for different children in the dataset and weight represents the weight values for different children in the dataset. However, one might be able to argue that weight doesn't tell us explain anything special about age that a child's height didn't already tell us. Since tall people tend to be heavier, the simple model above offers no way to tell whether weight is explaining variability in children's ages that height could not also explain. By adding height to the original model (in a model like this: $age = \beta_0 + \beta_1weight + \beta_2height + \epsilon$), you can address this issue. Adding height will mean that the slope for the weight-age relationship will become the unique relationship between weight and age that is not shared with height (since height is now in the model). Correspondingly, the slope for height in your new model will be the unique relationship between height and age that is not shared with weight. Typically in data analysis programs, constant refers to the intercept ($\beta_0$ in my example) and the "B =" lines refer to slopes (maybe you already know this, sorry if I'm speaking a too low a level). So you could fill in estimates for the intercept and slopes for the multiple regression model and it would look like this if the intercept was 5, the slope for weight was 3 and the slope for height was 7:

Model 2: $$age = 5 + 3*weight + 7*height + \epsilon$$

Here 3 would tell you that separate from whatever differences in age that both weight AND height explain, a child that ways 1 pound heavier is predicted to be one 3 months older (if age is in months and weight is in pounds). Similarly, over and above any shared explanatory power of weight and height, for each additional inch of height a child is expected to be 7 months older. Terms that mean the same as controlling for are: 'over and above', 'partialing out" or "holding constant". The reason your slopes are not the same from simple regression (models like model 1) to multiple regression (models like model 2) is because your independent variables are related and you are pulling out from the slope for weight (in this example) any part of that slope that is shared with height. The more related your independent variables are, the more your slopes will change from model 1 to model 2. When two IV's are related to each other this is called redundancy (there is also a particular type of redundancy called suppression, but you probably don't need to worry about that). I would suggest checking out a chapter on multiple regression in a well-reviewed statistics text book (you probably want to find one at the graduate level). (As a side note, redundancy is also called multi-collinearity and is sometimes taught as something to be wary of. This simple means that your independent variables are related, so it's not really that big of a deal,McClelland, Judd and Ryan, 2008 have a nice chapter on these issues in their statistics textbook.). Best of luck on your honor's thesis!