Take the 2-minute tour ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

In R, I have a data frame comprising a class label C (a factor) and two measurements, M1 and M2. How do I compute the correlation between M1 and M2 within each class?

Ideally, I'd get back a data frame with one row for each class and two columns: the class label C and the correlation.

share|improve this question
add comment

4 Answers 4

up vote 16 down vote accepted

The package plyr is the way to go.

Here is a simple solution:

xx <- data.frame(group = rep(1:4, 100), a = rnorm(400) , b = rnorm(400) )
head(xx)

require(plyr)
func <- function(xx)
{
return(data.frame(COR = cor(xx$a, xx$b)))
}

ddply(xx, .(group), func)

The output will be:

  group         COR
1     1  0.05152923
2     2 -0.15066838
3     3 -0.04717481
4     4  0.07899114
share|improve this answer
1  
(+1) Nice plyr package, isn't it? :) –  chl Oct 28 '10 at 8:47
    
This works great. Thanks for pointing out the plyr package! Could you please explain the ".(group)" syntax? –  NPE Oct 28 '10 at 14:31
2  
aix - sure. It means "split the data by the variable between .(), and on each subset perform the function". In order to have it include more variables, you should simply use this syntax: .(var1, var2, var3) . Which is like cutting your data by each combination of levels of var1, var2 and var3. And on each cut to perform your function. This package is maintained by Hadley (also th author of ggplot2), so I trust it will keep developing. –  Tal Galili Oct 29 '10 at 6:54
2  
Oh, and BTW, you could also use plyr with a parallel computing on several cores (almost automatically), see: r-statistics.com/2010/09/… –  Tal Galili Oct 29 '10 at 6:55
1  
That's a nice answer, but I'm astonished there isn't a built-in solution for this, something like cor(x, y, by=z) would be so intuitive... –  Waldir Leoncio Aug 1 '13 at 18:20
show 1 more comment

If you are inclined to use functions in the base package, you can use the by function, then reassemble the data:

xx <- data.frame(group = rep(1:4, 100), a = rnorm(400) , b = rnorm(400) )
head(xx)

# This returns a "by" object
result <- by(xx[,2:3], xx$group, function(x) {cor(x$a, x$b)})

# You get pretty close to what you want if you coerce it into a data frame via a matrix
result.dataframe <- as.data.frame(as.matrix(result))

# Add the group column from the row names
result.dataframe$C <- rownames(result)
share|improve this answer
1  
Nice, thanks! I've been experimenting with by, but couldn't figure out how to transform the result into a data frame. –  NPE Oct 28 '10 at 14:38
    
+1, I certainly appreciate the use of a base function. –  Patrick Coulombe Feb 14 at 3:59
add comment

Another example using base packages and Tal's example data:

DataCov <- do.call( rbind, lapply( split(xx, xx$group),
             function(x) data.frame(group=x$group[1], mCov=cov(x$a, x$b)) ) )
share|improve this answer
    
Elegant solution Joshue. Do you think there are cases when one solution is better then another? –  Tal Galili Oct 29 '10 at 16:05
2  
I think it's a matter of preference. My example is essentially what plyr does but it gives you finer control, though it's not nearly as clean. My opinion would change if one solution had a better time/memory profile. I haven't compared them though. –  Joshua Ulrich Oct 29 '10 at 16:08
add comment

Here's a similar method that will give you a table with the n's and p values for each correlation as well (rounded to 3 decimal places for convenience):

corrByGroup <- function(xx){
  return(data.frame(cbind(correl = round(rcorr(xx$a, xx$b)$r[1,2], digits=3),
                          n = rcorr(xx$a, xx$b)$n[1,2],
                          pvalue = round(rcorr(xx$a, xx$b)$P[1,2], digits=3))))
}
share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.