Interpreting results of a regression with categorical variable

Question

I am running regression on a dataset with 3 variables: n (an integer), Type (a categorical with levels A, B or C) and value (a numeric).

The result of

lm(Value ~ n + Type, data = dat) %>%
  summary()

is

Call:
lm(formula = Value ~ n + Type, data = dat)

Residuals:
     Min       1Q   Median       3Q      Max 
-0.45715 -0.06768 -0.00845  0.04665  0.63154 

Coefficients:
              Estimate Std. Error t value Pr(>|t|)    
(Intercept)  0.4496957  0.0350759  12.821  < 2e-16 ***
n            0.0005047  0.0010903   0.463    0.644    
TypeB        2.2800131  0.0346840  65.737  < 2e-16 ***
TypeC       -0.1854407  0.0346840  -5.347 3.86e-07 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 0.1645 on 131 degrees of freedom
Multiple R-squared:  0.9795,    Adjusted R-squared:  0.9791 
F-statistic:  2090 on 3 and 131 DF,  p-value: < 2.2e-16

If I'm not mistaken this means that for Type B the estimate for n is significantly higher than for Type A, and for Type C it is significantly lower than for Type A.

However, when I subset the data to the individual categories and do separate regressions:

lm(Value ~ n, data = dat[dat$Type == "A",]) %>%
  summary()

lm(Value ~ n, data = dat[dat$Type == "B",]) %>%
      summary()

lm(Value ~ n, data = dat[dat$Type == "C",]) %>%
      summary()

I get

Call:
lm(formula = Value ~ n, data = dat[dat$Type == "A", ])

Residuals:
      Min        1Q    Median        3Q       Max 
-0.136187 -0.061630  0.008786  0.049284  0.153302 

Coefficients:
             Estimate Std. Error t value Pr(>|t|)    
(Intercept) 0.4090728  0.0223533  18.300   <2e-16 ***
n           0.0022709  0.0008463   2.683   0.0103 *  
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 0.07373 on 43 degrees of freedom
Multiple R-squared:  0.1434,    Adjusted R-squared:  0.1235 
F-statistic: 7.201 on 1 and 43 DF,  p-value: 0.0103


Call:
lm(formula = Value ~ n, data = dat[dat$Type == "B", ])

Residuals:
    Min      1Q  Median      3Q     Max 
-0.4560 -0.2247 -0.0687  0.1733  0.6256 

Coefficients:
            Estimate Std. Error t value Pr(>|t|)    
(Intercept) 2.715986   0.083074   32.69   <2e-16 ***
n           0.001101   0.003145    0.35    0.728    
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 0.274 on 43 degrees of freedom
Multiple R-squared:  0.002844,  Adjusted R-squared:  -0.02035 
F-statistic: 0.1226 on 1 and 43 DF,  p-value: 0.7279

Call:
lm(formula = Value ~ n, data = dat[dat$Type == "C", ])

Residuals:
      Min        1Q    Median        3Q       Max 
-0.030407 -0.016048 -0.003995  0.014188  0.039227 

Coefficients:
              Estimate Std. Error t value Pr(>|t|)    
(Intercept)  0.3186009  0.0056277  56.613  < 2e-16 ***
n           -0.0018582  0.0002131  -8.721 4.63e-11 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 0.01856 on 43 degrees of freedom
Multiple R-squared:  0.6388,    Adjusted R-squared:  0.6304 
F-statistic: 76.06 on 1 and 43 DF,  p-value: 4.627e-11

In other words, for Type A the estimate is significantly higher than 0, for Type B, it is not significantly different from 0, and for Type C it is significantly lower than 0.

I'm trying to reconcile the following two sentences:

1. For Type B, the estimate for n is significantly higher than for Type A (from the original regression)

2. For Type A the estimate is significantly higher than 0, for Type B, it is not significantly different from 0 (from the individual regressions)

How can both these statements be true?

Answer 1

We can simulate some data, with three different slopes and intercepts, like your data, for type A we have intercept = 0, slope = 0, type B, intercept = 3, slope = 5 and type C, intercept =6 , slope = -4 .

library(ggplot2)
set.seed(222)
dat = data.frame(Type = rep(c("A","B","C"),each=10),n=runif(30))
dat$Value[dat$Type=="A"] = dat$n[dat$Type=="A"] + rnorm(10)
dat$Value[dat$Type=="B"] = 3+ 5*dat$n[dat$Type=="B"]
dat$Value[dat$Type=="C"] = 6 + -4*dat$n[dat$Type=="C"]

ggplot(dat,aes(x=n,y=Value,col=Type)) + 
geom_point() + geom_smooth(method="lm",se=FALSE)

When you fit this model:

lm(Value ~ n + Type, data = dat)

It basically saying Value can be explained by additive effect of Type and n. It fits a common slope for n (without regarding type) and adds a value depending on Type.

lm(Value ~ n + Type, data = dat)

Call:
lm(formula = Value ~ n + Type, data = dat)

Coefficients:
(Intercept)            n        TypeB        TypeC  
    -0.3142       1.1202       4.6077       3.5252 

You can work out the above by checking the prediction.

If you want to work out the different slopes in each type, you need to do:

lm(Value ~ n * Type, data = dat)

Call:
lm(formula = Value ~ n * Type, data = dat)

Coefficients:
(Intercept)            n        TypeB        TypeC      n:TypeB      n:TypeC  
    -0.5272       1.5583       3.5272       6.5272       3.4417      -5.5583

This fits an intercept value for each Type. In this case, the coefficient n is the coefficient for typeA and for type B, your coefficient for n will be 1.5583 + 3.4417 = 5 which is what we have simulated. Likewise for C, it will be 1.5583 - (-5.5583) = 6.