My question is pretty straightforward: It seems to me that using a correlation coefficient instead of simply directly calculating the coefficient of determination (using one set of observations as your predictors) will erroneously inflate the reliability estimation, by introducing a "hidden" linear regression into the model.
So, why is this done?
To help illustrate my point: Consider an examination of batting averages over the course of a baseball season. In order to test the reliability of the "batting average" statistic, we might randomly partition our data in half and then correlate each player's batting average between the two halves of the data.
However, a correlation here seems inappropriate, as it "hides" a linear regression in our analysis.
To see this, imagine that by some miracle, each player's batting average in the "first" half of the data varies from their batting average in the "second" half proportionally, by a fixed constant (this is exceedingly unlikely if we randomly partition our data, but it serves to illustrate the point). For such an arrangement, r (and thus r^2) is unity, because there's a perfectly linear relationship between the values (even though they may differ substantially!). From this, we conclude that our test is perfectly reliable. This seems pretty clearly erroneous.
On the other hand, if we simply take the sum of squared differences between each player's batting average in each half of the data, and then divide that by the variance of one half of the data (in essence, calculating a coefficient of determination for a model which naively predicts the batting average in the "second" half of the data to be the same as in the "first," rather than doing a linear regression), we clearly do not get a value of unity.
Edit #2: Another way to think of this, is that it seems that split-half reliability testing is basically 2-fold cross-validation using a linear regression as the model. My question is, what justification is there for the linear regression?