0
$\begingroup$

Let us define the $r^{2}$ as: $$ r^{2} = \left(\frac{\sum_{i=1}^{n}(obs_{i} - \overline{obs})(pred_{i}-\overline{pred})}{\sqrt{\sum_{i=1}^{n}(obs_{i} - \overline{obs})^{2}}\sqrt{\sum_{i=1}^{n}(pred_{i} - \overline{pred})^{2}}}\right)^{2}, $$ the amount of variance explained (AVE) as: $$ \mathrm{AVE} = 1-\frac{\sum_{i=1}^{n}(obs_{i} - pred_{i})^{2}}{\sum_{i=1}^{n}(obs_{i}-\overline{obs})^{2}}, $$ and the Lin's concordance coefficient ($\rho_{c}$) as: $$ \rho_{c} = \frac{2r\sigma_{pred}\sigma_{obs}}{\sigma_{obs}^{2}\sigma_{pred}^{2}+(\mu_{obs} - \mu_{pred})^{2}} = rC_{b}, $$ where $r$ is the Pearson's correlation coefficient, $\sigma$ is the variance ($r\sigma_{pred}\sigma_{obs}$ is the covariance between observed and predicted values) and $\mu$ is the mean. The $\rho_{c}$ reduces to $rC_{b}$ where $C_{b}$ is the bias correction factor defined as: \begin{equation} C_{b} = \frac{2}{v+1/v+u^{2}}, \end{equation} with $v = \sigma_{x}/\sigma_{y}$ being the scale shift and $u=(\mu_{x}-\mu_{y})/\sqrt{\sigma_{y}\sigma_{x}}$ being the location shift relative to the scale.

In my field (environmental science) there is a lot of confusion between the three. I often see people assessing the prediction accuracy of their model by the $r^{2}$. This does not make a lot of sense if the model is not a linear regression (e.g. a ML model) and I always suggest to use the AVE instead. Imagine a scatterplot between the predicted and measured values, the AVE shows the amount of variance explained by the model with respect to the 1:1 line, not to any regression line as it is for the $r^{2}$.

But then people might say that there is redundancy between the AVE and $\rho_{c}$, because both show the agreement of predicted versus observed with respect to the 1:1 line. From the equation above it appears that the $\rho_{c}$ is simply a Pearson's $r$ times a bias correction factor.

I made a few numerical tests to try to understand how the AVE and the $\rho_{c}$ relate. Under perfect correlation (i.e. $r=1$) and increasing shift with respect to the 1:1 line, the $AVE = log(\rho_{c})+b$ where $b$ is a constant. Both the AVE and the $\rho_{c}$ almost linearly decrease with increase of the shift from the 1:1 line.

Under three different degrees of correlation, for weak ($r=0.22$), moderate ($r=0.75$) and strong ($r=0.98$) correlation between the predicted and observed values, and increasing shift with respect to the 1:1 line, I obtain the following figure. The low values of AVE and $\rho_{c}$ are of course related to large shift from the 1:1 line.

enter image description here

My questions are as follows:

  1. Is there redundancy between the AVE and $\rho_{c}$ when one wishes to assess the prediction accuracy of a model with respect to some observations?

  2. Is there a direct relation between the AVE and the $\rho_{c}$ that I have missed. How to explain the figure above?

  3. Is there a case showing a high $\rho_{c}$ value and low AVE?

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.