Averaging correlation values Let's say I test how variable Y depends on variable X under different experimental conditions and obtain the following graph:

The dash lines in the graph above represent linear regression for each data series (experimental setup) and the numbers in the legend denote the Pearson correlation of each data series.
I would like to calculate the "average correlation" (or "mean correlation") between X and Y. May I simply average the r values? What about the "average determination criterion", $R^2$? Should I calculate the average r and than take the square of that value or should I compute the average of individual $R^2$'s?
 A: For Pearson correlation coefficients, it is generally appropriate to transform the r values using a Fisher z transformation. Then average the z-values and convert the average back to an r value. 
I imagine it would be fine for a Spearman coefficient as well.
Here's a paper and the wikipedia entry. 
A: The average correlation can be meaningul.  Also consider the distribution of correlations (for example, plot a histogram).
But as I understand it, for each individual you have some ranking of $n$ items plus predicted rankings of those items for that individual, and you're looking at the correlation between an individual's rankings and the predicted ones.
In this case, it may be that the correlation is not the best measure of how well the algorithm is making predictions.  For example, imagine that the algorithm gets the first 100 items perfectly and the next 200 items totally messed up, vs the opposite.  It could be that you care only about the quality of the top rankings.  In this case, you might look at the sum of the absolute differences between the individual's ranking and the predicted ranking, but only among the individual's top $m$ items.
A: The simple way is to add a categorical variable $z$ to identify the different experimental conditions and include it in your model along with an "interaction" with $x$; that is, $y \sim z + x\#z$.  This conducts all five regressions at once.  Its $R^2$ is what you want.
To see why averaging individual $R$ values may be wrong, suppose the direction of the slope is reversed in some of the experimental conditions. You would average a bunch of 1's and -1's out to around 0, which wouldn't reflect the quality of any of the fits. To see why averaging $R^2$ (or any fixed transformation thereof) is not right, suppose that in most experimental conditions you had only two observations, so that their $R^2$ all equal $1$, but in one experiment you had a hundred observations with $R^2=0$. The average $R^2$ of almost 1 would not correctly reflect the situation.
A: What about using mean squared predicted eror (MSPE) for the performance of the algorithm?  This is a standard approach to what you are trying to do, if you are trying to compare predictive performance among a set of algorithms.
