9
$\begingroup$

I will give my examples with R calls. First a simple example of a linear regression with a dependent variable 'lifespan', and two continuous explanatory variables.

data.frame(height=runif(4000,160,200))->human.life
human.life$weight=runif(4000,50,120)
human.life$lifespan=sample(45:90,4000,replace=TRUE)
summary(lm(lifespan~1+height+weight,data=human.life))

Call:
lm(formula = lifespan ~ 1 + height + weight, data = human.life)

Residuals:
Min       1Q   Median       3Q      Max 
-23.0257 -11.9124  -0.0565  11.3755  23.8591 

Coefficients:
             Estimate Std. Error t value Pr(>|t|)    
(Intercept) 63.635709   3.486426  18.252   <2e-16 ***
height       0.007485   0.018665   0.401   0.6884    
weight       0.024544   0.010428   2.354   0.0186 *  
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1 

Residual standard error: 13.41 on 3997 degrees of freedom
Multiple R-squared: 0.001425,   Adjusted R-squared: 0.0009257 
F-statistic: 2.853 on 2 and 3997 DF,  p-value: 0.05781

In order to find the estimate of 'lifespan' when the value of 'weight' is 1, I add (Intercept)+height=63.64319

Now what if I have a similar data frame, but one where one of the explanatory variables is categorical?

data.frame(animal=rep(c("dog","fox","pig","wolf"),1000))->animal.life
animal.life$weight=runif(4000,8,50)
animal.life$lifespan=sample(1:10,replace=TRUE)
summary(lm(lifespan~1+animal+weight,data=animal.life))

Call:
lm(formula = lifespan ~ 1 + animal + weight, data = animal.life)

Residuals:
Min      1Q  Median      3Q     Max 
-4.7677 -2.7796 -0.1025  3.1972  4.3691 

Coefficients:
            Estimate Std. Error t value Pr(>|t|)    
(Intercept) 5.565556   0.145851  38.159  < 2e-16 ***
animalfox   0.806634   0.131198   6.148  8.6e-10 ***
animalpig   0.010635   0.131259   0.081   0.9354    
animalwolf  0.806650   0.131198   6.148  8.6e-10 ***
weight      0.007946   0.003815   2.083   0.0373 *  
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1 

Residual standard error: 2.933 on 3995 degrees of freedom
Multiple R-squared: 0.01933,    Adjusted R-squared: 0.01835 
F-statistic: 19.69 on 4 and 3995 DF,  p-value: 4.625e-16

In this case, to find the estimate of 'lifespan' when the value of 'weight' is 1, should I add each of the coefficients for 'animal' to the intercept: (Intercept)+animalfox+animalpig+animalwolf? Or what is the proper way to do this?

Thanks Sverre

Improve this question
$\endgroup$
6
  • $\begingroup$ dollar signs make you enter an equation environment and is why things are randomly getting italicized. $\endgroup$
    – Macro
    Jul 21, 2011 at 22:22
  • $\begingroup$ formatting: to get code, indent by 4 spaces. $\endgroup$
    – wolf.rauch
    Jul 21, 2011 at 22:38
  • $\begingroup$ if you use the 4 spaces indenting, you can put stars and dollar signs in there and they will show up as such. if you use them outside the code formatting, they will be treated like markup. If you don't want a complete code line, use backticks: this is code with a $ and * $\endgroup$
    – wolf.rauch
    Jul 21, 2011 at 22:54
  • 1
    $\begingroup$ It is good that you have used a reproducible example. You could make the example even better by including set.seed(1) (or whatever number you like) before running random number generation, so that everybody gets exactly the same results as you (not that it matters much in this case, though). $\endgroup$
    – wolf.rauch
    Jul 21, 2011 at 23:08
  • $\begingroup$ Just two small comments. You wrote "In order to find the estimate of 'lifespan' when the value of 'weight' is 1, I add (Intercept)+height=63.64319". Note that this is the estimated average lifespan when weight is = 1 and height = 0. That is probably not very meaningful. Also, predictions outside of the range of the observed values of the independent variables need to be treated with caution (weight is between 50 and 120, so weight = 1 is also not very meaningful). Just some side-notes and probably things you already knew. But just in case ... $\endgroup$
    – Wolfgang
    Jul 22, 2011 at 12:43

3 Answers 3

Reset to default
12
$\begingroup$

No, you shouldn't add all of the coefficients together. You essentially have the model

$$ {\rm lifespan} = \beta_{0} + \beta_{1} \cdot {\rm fox} + \beta_{2} \cdot {\rm pig} + \beta_{3} \cdot {\rm wolf} + \beta_{4} \cdot {\rm weight} + \varepsilon $$

where, for example, ${\rm pig} = 1$ if the animal was a pig and 0 otherwise. So, to calculate $\beta_{0} + \beta_{1} + \beta_{2} + \beta_{3} + \beta_{4}$ as you've suggested for getting the overall average when ${\rm weight}=1$ is like saying "if you were a pig, a wolf, and a fox, and your weight was 1, what is your expected lifespan?". Clearly since each animal is only one of those things, that doesn't make much sense.

You will have to do this separately for each animal. For example, $\beta_{0} + \beta_{2} + \beta_{4}$ is the expected lifespan for a pig when its weight is 1.

Improve this answer
$\endgroup$
5
  • $\begingroup$ Macro: I understand. Would it makes sense, then, to find the average coefficient for the levels in 'animal'? In other words do (Intercept)+(animalfox+animalpig+animalwolf)/3. Or is that valid only when there's an equal number of observations for each animal in the data set? $\endgroup$
    – Sverre
    Jul 21, 2011 at 22:45
  • $\begingroup$ I think you're right - that would only be valid if there are an equal number in each group. You could weight them proportional to how heavily each group is represented in the sample if you insist on summarizing it into one number. $\endgroup$
    – Macro
    Jul 21, 2011 at 23:25
  • $\begingroup$ What would be the proper way to weight them? The reason I am "insisting" on summarizing it into one number is that I want to draw the regression line for only one of the variables on a scatter plot. For example, in the human.life example above, I would draw the regression line for 'weight' by specifying the line's intercept ((Intercept)+height=63.64319) with its coefficient (0.024544). That's just a little trickier in the animal.life case. $\endgroup$
    – Sverre
    Jul 22, 2011 at 0:01
  • $\begingroup$ Thinking more about it, I don't know how any kind of average would be interpretable. You could just draw three parallel regression lines on one plot couldn't you? Also, it seems to me that the effect of "weight" would be different for each animal, in which case you should have animal interacting with weight, which would lead to three completely different regression lines for each animal. $\endgroup$
    – Macro
    Jul 22, 2011 at 0:05
  • $\begingroup$ But in a case where the variable for 'animal' and the variable for 'weight' are both significant, but their interaction is not, I wouldn't even include the interaction in the model. The multiple regression estimates the effect of 'weight' independent of what the value for 'animal' is. $\endgroup$
    – Sverre
    Jul 22, 2011 at 0:14
4
$\begingroup$

The simplest thing to do is to use the predict function on the lm object, then it take care of many of the details like converting a factor to the right values to add together. If you are trying to understand the pieces that go into the prediction then set type='terms' and it will show the individual pieces that add together make your prediction.

Note also that how a factor is converted to variables depends on some options, the default will choose a baseline group to compare the other groups to, but you can also set it to an average and differences from that average (or other comparisons of interest).

Improve this answer
$\endgroup$
2
  • $\begingroup$ Using predict.lm() is a good solution for an lm object. Unfortunately, I am actually fitting an lmer object to my data, for which no predict() function extracting individual terms exists. Did I understand you correctly that the alternative method you're suggesting would set the intercept to be the average value, instead of a baseline (where all continuous predictors are set to 0, and one value of categorical predictors is chosen)? If so, I would like to know how to do that. Then I could just draw my regression line as the intercept of the model + the coefficient of my predictor. $\endgroup$
    – Sverre
    Jul 22, 2011 at 20:43
  • $\begingroup$ Look at ?contrasts, ?C, ?contr.sum, and the contrasts part of ?options. $\endgroup$
    – Greg Snow
    Jul 23, 2011 at 13:17
2
$\begingroup$

If you want the average lifespan when weight is 1 then you can just take out "animal" in this call:

lm(formula = lifespan ~ 1 + animal + weight, data = animal.life)
Improve this answer
$\endgroup$
1
  • $\begingroup$ I don't understand how this can be right. If I take out one of the predictors ('animal'), then the intercept, coefficient of 'weight', and the error estimates all change. Also, I am not trying to find out what the actual average lifespan for weight 1 in the data is, just what the model predicts it should be. $\endgroup$
    – Sverre
    Jul 22, 2011 at 20:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.