Let us define the $r^{2}$ as: $$ r^{2} = \left(\frac{\sum_{i=1}^{n}(obs_{i} - \overline{obs})(pred_{i}-\overline{pred})}{\sqrt{\sum_{i=1}^{n}(obs_{i} - \overline{obs})^{2}}\sqrt{\sum_{i=1}^{n}(pred_{i} - \overline{pred})^{2}}}\right)^{2}, $$ the amount of variance explained (AVE) as: $$ \mathrm{AVE} = 1-\frac{\sum_{i=1}^{n}(obs_{i} - pred_{i})^{2}}{\sum_{i=1}^{n}(obs_{i}-\overline{obs})^{2}}, $$ and the Lin's concordance coefficient ($\rho_{c}$) as: $$ \rho_{c} = \frac{2r\sigma_{pred}\sigma_{obs}}{\sigma_{obs}^{2}\sigma_{pred}^{2}+(\mu_{obs} - \mu_{pred})^{2}} = rC_{b}, $$ where $r$ is the Pearson's correlation coefficient, $\sigma$ is the variance ($r\sigma_{pred}\sigma_{obs}$ is the covariance between observed and predicted values) and $\mu$ is the mean. The $\rho_{c}$ reduces to $rC_{b}$ where $C_{b}$ is the bias correction factor defined as: \begin{equation} C_{b} = \frac{2}{v+1/v+u^{2}}, \end{equation} with $v = \sigma_{x}/\sigma_{y}$ being the scale shift and $u=(\mu_{x}-\mu_{y})/\sqrt{\sigma_{y}\sigma_{x}}$ being the location shift relative to the scale.

In my field (environmental science) there is a lot of confusion between the three. I often see people assessing the prediction accuracy of their model by the $r^{2}$. This does not make a lot of sense if the model is not a linear regression (e.g. a ML model) and I always suggest to use the AVE instead. Imagine a scatterplot between the predicted and measured values, the AVE shows the amount of variance explained by the model with respect to the 1:1 line, not to any regression line as it is for the $r^{2}$.

But then people might say that there is redundancy between the AVE and $\rho_{c}$, because both show the agreement of predicted versus observed with respect to the 1:1 line. From the equation above it appears that the $\rho_{c}$ is simply a Pearson's $r$ times a bias correction factor.

I made a few numerical tests to try to understand how the AVE and the $\rho_{c}$ relate. Under perfect correlation (i.e. $r=1$) and increasing shift with respect to the 1:1 line, the $AVE = log(\rho_{c})+b$ where $b$ is a constant. Both the AVE and the $\rho_{c}$ almost linearly decrease with increase of the shift from the 1:1 line.

Under three different degrees of correlation, for weak ($r=0.22$), moderate ($r=0.75$) and strong ($r=0.98$) correlation between the predicted and observed values, and increasing shift with respect to the 1:1 line, I obtain the following figure. The low values of AVE and $\rho_{c}$ are of course related to large shift from the 1:1 line.

enter image description here

My questions are as follows:

  1. Is there redundancy between the AVE and $\rho_{c}$ when one wishes to assess the prediction accuracy of a model with respect to some observations?

  2. Is there a direct relation between the AVE and the $\rho_{c}$ that I have missed. How to explain the figure above?

  3. Is there a case showing a high $\rho_{c}$ value and low AVE?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.