Regression coefficients by group in R?

Question

I have a dataframe with a group variable GRP (ranging from 1-100) and an X and Y for each one. I'd like to get a list of the regression intercepts and slopes for lm(Y~X) within each group. The intercepts and slopes don't need to be in the same dataframe.

Any suggestions? R beginner here, so simplicity would be great!

Answer 1

data.table also has great tools for solving problems such as this:

library(data.table)
set.seed(1)
dat <- data.table(x=runif(100), y=runif(100), grp=rep(1:2,50))
dat[,coef(lm(y~x)),by=grp]

The first row in each group is the intercept, and the second row is the coefficient:

     grp         V1
[1,]   1  0.5991761
[2,]   1 -0.1350489
[3,]   2  0.4401174
[4,]   2  0.1400153

If you'de rather have a wide data.frame, that just takes a little more specification:

dat[,list(intercept=coef(lm(y~x))[1], coef=coef(lm(y~x))[2]),by=grp]
     grp intercept       coef
[1,]   1 0.5991761 -0.1350489
[2,]   2 0.4401174  0.1400153

Or you could put it even more succinctly as:

    dat[,as.list(coef(lm(y~x))),by=grp]
    (Intercept)          x
    1:   1   0.5991761 -0.1350489
    2:   2   0.4401174  0.1400153

Answer 2

The responses by @Henry and @Zach both work, but I think the most straight-forward way to do what you want is to use lmList in the nlme package:

dat <- data.frame(
  GRP = sample(c("A","B","C"), 100, replace=TRUE), 
  X = runif(100), 
  Y = runif(100)
)
require(nlme)
lmList(Y ~ X | GRP, data=dat)

Answer 3

Adapting from help("by"), this example may meet your needs

mydf <- data.frame( GRP = rep(c("A","B","C"), each=100), X = rep(1:100,3), 
                    Y = rep(c(2,4,8),each=100) + 
                        rep(c(4,2,1),each=100) * rep(1:100,3) + rnorm(300))   
by(mydf, mydf$GRP, function(z) lm(Y ~ X, data = z))

Answer 4

If you use package "tidyr" you could do the following

library(data.table)
library(tidyr)
set.seed(1)
dat <- data.table(x=runif(100), y=runif(100), grp=rep(1:2,50))

ncoefs <- 1
dat <- dat[, coef( lm(y ~ x) ), by = grp]
dat[, est := rep( c("intercept", "coef"), .N/(ncoefs + 1)) ]
dat <- dat %>% spread(est, V1)

The result is

   grp       coef intercept
1:   1 -0.1350489 0.5991761
2:   2  0.1400153 0.4401174

This method is easy to scale up and must be faster than estimating the regression for each coefficient.