17
votes
$\begingroup$

In R, I have a data frame comprising a class label C (a factor) and two measurements, M1 and M2. How do I compute the correlation between M1 and M2 within each class?

Ideally, I'd get back a data frame with one row for each class and two columns: the class label C and the correlation.

$\endgroup$

6 Answers 6

Reset to default
20
votes
$\begingroup$

The package plyr is the way to go.

Here is a simple solution:

xx <- data.frame(group = rep(1:4, 100), a = rnorm(400) , b = rnorm(400) )
head(xx)

require(plyr)
func <- function(xx)
{
return(data.frame(COR = cor(xx$a, xx$b)))
}

ddply(xx, .(group), func)

The output will be:

  group         COR
1     1  0.05152923
2     2 -0.15066838
3     3 -0.04717481
4     4  0.07899114
$\endgroup$
6
  • 1
    $\begingroup$ (+1) Nice plyr package, isn't it? :) $\endgroup$
    – chl
    Commented Oct 28, 2010 at 8:47
  • $\begingroup$ This works great. Thanks for pointing out the plyr package! Could you please explain the ".(group)" syntax? $\endgroup$
    – NPE
    Commented Oct 28, 2010 at 14:31
  • 2
    $\begingroup$ aix - sure. It means "split the data by the variable between .(), and on each subset perform the function". In order to have it include more variables, you should simply use this syntax: .(var1, var2, var3) . Which is like cutting your data by each combination of levels of var1, var2 and var3. And on each cut to perform your function. This package is maintained by Hadley (also th author of ggplot2), so I trust it will keep developing. $\endgroup$
    – Tal Galili
    Commented Oct 29, 2010 at 6:54
  • 2
    $\begingroup$ Oh, and BTW, you could also use plyr with a parallel computing on several cores (almost automatically), see: r-statistics.com/2010/09/… $\endgroup$
    – Tal Galili
    Commented Oct 29, 2010 at 6:55
  • 1
    $\begingroup$ That's a nice answer, but I'm astonished there isn't a built-in solution for this, something like cor(x, y, by=z) would be so intuitive... $\endgroup$
    – Waldir Leoncio
    Commented Aug 1, 2013 at 18:20
12
votes
$\begingroup$

If you are inclined to use functions in the base package, you can use the by function, then reassemble the data:

xx <- data.frame(group = rep(1:4, 100), a = rnorm(400) , b = rnorm(400) )
head(xx)

# This returns a "by" object
result <- by(xx[,2:3], xx$group, function(x) {cor(x$a, x$b)})

# You get pretty close to what you want if you coerce it into a data frame via a matrix
result.dataframe <- as.data.frame(as.matrix(result))

# Add the group column from the row names
result.dataframe$C <- rownames(result)
$\endgroup$
1
  • 1
    $\begingroup$ Nice, thanks! I've been experimenting with by, but couldn't figure out how to transform the result into a data frame. $\endgroup$
    – NPE
    Commented Oct 28, 2010 at 14:38
9
votes
$\begingroup$

Another example using base packages and Tal's example data:

DataCov <- do.call( rbind, lapply( split(xx, xx$group),
             function(x) data.frame(group=x$group[1], mCov=cov(x$a, x$b)) ) )
$\endgroup$
3
  • $\begingroup$ Elegant solution Joshue. Do you think there are cases when one solution is better then another? $\endgroup$
    – Tal Galili
    Commented Oct 29, 2010 at 16:05
  • 2
    $\begingroup$ I think it's a matter of preference. My example is essentially what plyr does but it gives you finer control, though it's not nearly as clean. My opinion would change if one solution had a better time/memory profile. I haven't compared them though. $\endgroup$
    – Joshua Ulrich
    Commented Oct 29, 2010 at 16:08
  • $\begingroup$ How does this return the correlation? $\endgroup$
    – user14650
    Commented Aug 14, 2015 at 10:44
2
votes
$\begingroup$

Using data.table is shorter than dplyr

dt <- data.table(xx)
dtCor <- dt[, .(mCor = cor(M1,M2)), by=C]
$\endgroup$
0
votes
$\begingroup$

Here's a similar method that will give you a table with the n's and p values for each correlation as well (rounded to 3 decimal places for convenience):

library(Hmisc)
corrByGroup <- function(xx){
  return(data.frame(cbind(correl = round(rcorr(xx$a, xx$b)$r[1,2], digits=3),
                          n = rcorr(xx$a, xx$b)$n[1,2],
                          pvalue = round(rcorr(xx$a, xx$b)$P[1,2], digits=3))))
}
$\endgroup$
0
votes
$\begingroup$

Here's a more modern solution, using the dplyr package (which didn't yet exist when the question was asked):

Construct the input:

xx <- data.frame(group = rep(1:4, 100), a = rnorm(400) , b = rnorm(400) )

Compute the correlations:

library(dplyr)
xx %>%
  group_by(group) %>%
  summarize(COR=cor(a,b))

The output:

Source: local data frame [4 x 2]

  group         COR
  (int)       (dbl)
1     1  0.05112400
2     2  0.14203033
3     3 -0.02334135
4     4  0.10626273
$\endgroup$

Not the answer you're looking for? Browse other questions tagged or ask your own question.