On round robin $R^2$ values for three methods

Question

Introduction: In comparative studies where we plot two methods, $\left\{M_1,M_2\right\}$ we obtain an explained fraction $R^2$, between those studies. This give us no clue as to which method is better. However, if we have three or more methods, hints as to how good each method is begin to emerge. It would be a great contribution to the literature if we could say something about which method is superior based on those $R^2$ values. The desired $R^2$ values are what the explained fraction would be for pair-wise repeat measures for each method taken separately.

The problem: Suppose we have three methods; $\left\{M_1,M_2,M_3\right\}$ of measuring something. Now suppose that they are correlated $\left\{M_1,M_2\right\}\to R_{1,2}^2,\left\{M_2,M_3\right\}\to R_{2,3}^2,\left\{M_1,M_3\right\}\to R_{1,3}^2$, then there may exist some way of separating effects so that we can obtain $\left\{M_1\right\}\to R_1^2,\left\{M_2\right\}\to R_2^2,\left\{M_3\right\}\to R_3^2$

This is clearer if we put in numbers: Let $R_{1,2}^2=0.7,R_{2,3}^2=0.8,R_{1,3}^2=0.9$, then as $\frac{1}{2} \left(R_{1,3}^2+R_{2,3}^2\right)=0.85$

$\frac{1}{2} \left(R_{1,2}^2+R_{1,3}^2\right)=0.8$

$\frac{1}{2} \left(R_{1,2}^2+R_{2,3}^2\right)=0.75$

This suggests that $R_2^2<R_1^2<R_3^2$, so that some information as to how these quantities relate to each other likely exists. Thus the question is What are $R_1^2,R_2^2$ and $R_3^2$ numerically? Or, failing that How far can we go toward characterizing the relationship between $R_1^2,R_2^2$ and $R_3^2$?

Some thoughts on the question. It is unlikely to be so simple that one can treat $R^2$ as additive explained fractions (as covariance is nonzero, I would guess). If one could then for our example data, we might have

$R_1^2\approx R_{1,2}^2+R_{1,3}^2-R_{2,3}^2=0.8$

$R_2^2\approx R_{1,2}^2-R_{1,3}^2+R_{2,3}^2=0.6$

$R_3^2\approx -R_{1,2}^2+R_{1,3}^2+R_{2,3}^2=1.0$

Possible hint: For error propagation with $n=3$, one obtains for total variance

$\sigma _f^2=2 a_1 a_2 a_3 \sigma _1 \sigma _2 \sigma _3 \left(\frac{\rho _{1,2}}{a_3 \sigma _3}+\frac{\rho _{1,3}}{a_2 \sigma _2}+\frac{\rho _{2,3}}{a_1 \sigma _1}\right)+\left(a_1^2 \sigma _1^2+a_2^2 \sigma _2^2+a_3^2 \sigma _3^2\right)$

where the first term of the two part sum represents covariance, and the second variance and where $R$ in my notation is $\rho$.

Right now, I find nothing in the literature on this subject. Although there has been a paper on the confidence interval of an $R_{1,2},R_{1,3}$ difference.

Answer 1

I think I understand your question now, and the following should be closer to what you are after. To start, I will use the example from your comment.

Assume we have a set of machines that measure blood pressure, where a measurement from machine $k$ is given by

$$M_k[x] = a_kx + b_k + \sigma_k\epsilon$$

where $x$ is the (unknown) blood pressure, $(a_k,b_k,\sigma_k)$ are machine-dependent constants, and $\epsilon$ is "noise".

For simplicity, I will ignore finite-sample effects. The standard "regression" assumptions are then $$\langle\epsilon\rangle=0, \sigma_{x,\epsilon}=0, \sigma_{\epsilon_i\epsilon_j}=\delta_{i,j}$$ where angle brackets denote an average and $\sigma$'s are covariances.

Then we have

$$R_k = \frac{a_k^2\sigma_x^2}{a_k^2\sigma_x^2+\sigma_k^2}$$

and

$$R_{ij}^2 = \frac{a_i^2a_j^2\sigma_x^4}{(a_i^2\sigma_x^2+\sigma_i^2)(a_j^2\sigma_x^2+\sigma_j^2)} = R_iR_j$$

Now if we have three machines $M_1,M_2,M_3$, and $R_{ij}^4 = R_i^2R_j^2$ for any pair of them, this implies

$$R_k^4 = \frac{R_{ik}^4R_{jk}^4}{R_{ij}^4}$$

for any distinct $i,j,k$.

So expanding out, and using symmetry of $R_{ij}$, the answer is then

$$R_1^4 = \frac{R_{12}^4R_{13}^4}{R_{23}^4}, R_2^4 = \frac{R_{12}^4R_{23}^4}{R_{13}^4}, R_3^4 = \frac{R_{13}^4R_{23}^4}{R_{12}^4}$$

I checked this out numerically for finite samples, and it seems to hold reasonably (i.e. small error for large sample sizes).

Hope this helps!