0
$\begingroup$

I have a question regarding the interpretation of a linear regression output. In my data I have one independent categorical variable (condition) with five values which I represented as four dummy variables (one, two, three, four). Does the output mean that each of the possible four conditions (e.g. one) is compared to the condition five (the excluded one)?

output

What happens when I exclude four dummy variables and only run the regression with one dummy variable? Does it mean that category one is compared to the rest four possibilities taken together? Does a model like this make sense?

output2

$\endgroup$

1 Answer 1

1
$\begingroup$

Yes, you are exactly correct. The intercept is the mean value of the response variable for the reference condition. When you include dummies for conditions 1 through 4, the intercept is the mean response for condition 5, and the regression coefficients for dummies 1 through 4 are the differences in mean response for each condition relative to condition 5.

In your second output, which includes only dummy 1, the intercept is the mean response across conditions 2 through 5, and the regression coefficient for condition 1 is the difference in mean response for condition 1 versus the others collectively (weighted by sample size).

It is a conceptual question whether the latter analysis tells you anything. There might be cases where the result is valuable. But your initial results give you more information (at the cost of estimating more parameters and likely dealing with more collinearity). You can see there that mean response is higher for condition 1 is lower than for condition 5 but higher than for condition 4.

$\endgroup$
6
  • $\begingroup$ Thank you for your answer! I do not quite get what you mean by 'You can see there that mean response is higher for condition 1 is lower than for condition 5 but higher than for condition 4.' though. Do you mean that 1 (-24.6282) is lower than intercept (86.7586) but higher than 4 (-37.1961)? $\endgroup$ Apr 6, 2020 at 19:26
  • $\begingroup$ In the first output, the intercept you that mean response in the reference condition (5) is 86.8. You could write this as E(Y) = 86.8 - 24.6 * 0 - 25.4 * 0 - 29.1 * 0 - 37.2 * 0 E(Y) = 86.8 because, in the reference condition, the dummies all equal 0. In condition 1, this becomes: E(Y) = 86.8 - 24.6 * 1 - 25.4 * 0 - 29.1 * 0 - 37.2 * 0 E(Y) = 86.8 - 24.6 = 62.2 In condition 5, this becomes: E(Y) = 86.8 - 24.6 * 0 - 25.4 * 0 - 29.1 * 0 - 37.2 * 1 E(Y) = 86.8 - 37.2 = 49.6 So the expected value or mean of the response variable is lower in Condition 5 than in Condition 1. $\endgroup$
    – Ed Rigdon
    Apr 6, 2020 at 19:39
  • $\begingroup$ This is really great explanation, thank you! May be you could comment on a follow up question? Are the differences in mean response for each condition (one to four) independent from each other? I do not quite understand whether in my case the dummies are treated as independent variables in multiple regression or not, since they just represent different values of one categorical variable. I've read that the effect of one variable contributes significantly (or not) to the regression after the effect of another one is taken into account (assuming there are two indep. var). What is the case here? $\endgroup$ Apr 7, 2020 at 15:25
  • $\begingroup$ Not sure what you mean by "independent from each other." Only one effect is active at a time, since the classes are mutually exclusive, so the effects are not additive, except when comparing the value for a given dummy vs the value for the reference condition (when all dummies = 0). But the dummy variables are certainly correlated, with the strength of correlation being a declining function of the total number of categories (modified by the precise distribution of cases across categories). $\endgroup$
    – Ed Rigdon
    Apr 7, 2020 at 15:38
  • $\begingroup$ Ok, I think I understnd it now. However, is it multiple or simple regression? Should be multiple because I have more than one independent variable. But isn't it still simple regression since the dummies collectively represent one categorical variable? (In all explanations I could find there is always different variables each represented as one dummy with only two values, which is a different case) $\endgroup$ Apr 7, 2020 at 15:53

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.