6
$\begingroup$

Both can be used for an assessment of model accuracy, but what is the difference?

formula for coefficient of determination, or R²:

formula for coefficient of determination (R²)

with: SSres= sum (yi - fi)² and SStot = sum (yi - ymean)²

y = observed values (for model evaluation), f = modelled/predicted values.

source:wikipedia

formula for Nash-Sutcliffe efficiency:

NSE_formula

source: journal article "Model evaluation guidelines for systematic quantification of accuracy in watershed simulations", Moriasi et al. 2007

Do I miss something here or are the formulas identical? Sorry for my expression.

$\endgroup$
2
  • $\begingroup$ I don't think there is any ... looks same to me $\endgroup$ May 3, 2016 at 0:33
  • 1
    $\begingroup$ @AbdulRaheem, although the formulas may look the same, they appear to have different meanings. Could you therefore elaborate on your answer and perhaps supply some authoritative reference(s)? $\endgroup$
    – whuber
    May 3, 2016 at 1:00

2 Answers 2

11
$\begingroup$

The Nash-Sutcliffe model efficiency coefficient is nearly identical to the coefficient of determination. The primary difference is how it is used.

The coefficient of determination ($R^2$) is a measure of the goodness of fit of a statistical model.

\begin{equation} \begin{aligned} R^2 = 1 - \frac{\sum (y_i - \hat{y_i})^2}{\sum (y_i - \bar{y})^2} \end{aligned} \end{equation}

Where $y_i$ are the observed values of the variable of interest, $\hat{y_i}$ are the predicted values, and $\bar{y}$ is the mean of the observations. For example, if we have a set of obervations $x_i$ and $y_i$, we might assume a linear model $y=ax + b$ to predict this relationship, resulting in set of predicted values, $\hat{y_i}$. The smallest $R^2$ occurs when there is no correlation between $x$ and $y$ and the best prediction is to assume $b=\bar{y}$ and $m=0$. This corresponds to an $R^2$ value of 0, which is the lower limit of $R^2$ because the the sum of squares of the rediduals, $\sum (y_i - \hat{y_i})^2$, will never be greater than the total sum of squares, $\sum (y_i - \bar{y})^2$.

The Nash-Sutcliffe model efficiency coefficient ($E$) is used to quantify how well a model simulation can predict the outcome variable.

\begin{equation} \begin{aligned} E = 1 - \frac{\sum (y_i - y_{i,sim})^2}{\sum (y_i - \bar{y})^2} \end{aligned} \end{equation}

The variables are the same as described above, but $y_{i,sim}$ are the predictions from the simulation (instead of the $\hat{y_i}$ from a statistical model). The model may be calibrated, but the predicted values of the outcome variable $y_{i,sim}$ are not inferred from the observed values. Unlike with a statistical model, the sum of squares of the model error, $\sum (y_i - y_{i,sim})^2$, may be greater than the total sum of squares, $\sum (y_i - \bar{y})^2$, and the coefficient can therefore be negative.

$\endgroup$
5
$\begingroup$

This answer is directly copied from Krause et al., (2005) Krause P, Boyle DP, Bäse F. 2005. Comparison of different efficiency criteria for hydrological model assessment. Advances in Geosciences 5 (5): 89–97 DOI: 10.5194/adgeo-5-89-2005

The coefficient of determination r2 is defined as the squared value of the coefficient of correlation. enter image description here

R2 can also be expressed as the squared ratio between the covariance and the multiplied standard deviations of the observed and predicted values. Therefore it estimates the combined dispersion against the single dispersion of the observed and predicted series. The range of r2 lies between 0 and 1 which describes how much of the observed dispersion is explained by the prediction. A value of zero means no correlation at all whereas a value of 1 means that the dispersion of the prediction is equal to that of the observation. The fact that only the dispersion is quantified is one of the ma jor drawbacks of r2 if it is considered alone. A model which systematically over- or underpredicts all the time will still result in good r2 values close to 1.0 even if all predictions were wrong. If r2 is used for model validation it therefore is advisable to take into account additional information which can cope with that problem. Such information is provided by the gradient b and the intercept a of the regression on which r2 is based. For a good agreement the intercept a should be close to zero which means that an observed runoff of zero would also result in a prediction near zero and the gradient b should be close to one.

The efficiency E proposed by Nash and Sutcliffe (1970) is defined as one minus the sum of the absolute squared differences between the predicted and observed values normalized by the variance of the observed values during the period un der investigation. It is calculated as: enter image description here

The normalization of the variance of the observation series results in relatively higher values of E in catchments with higher dynamics and lower values of E in catchments with lower dynamics. To obtain comparable values of E in a catchment with lower dynamics the prediction has to be bet- ter than in a basin with high dynamics. The range of E lies between 1.0 (perfect fit) and −∞. An efficiency of lower than zero indicates that the mean value of the observed time series would have been a better predictor than the model. The largest disadvantage of the Nash-Sutcliffe efficiency is the fact that the differences between the observed and predicted values are calculated as squared values. As a re- sult larger values in a time series are strongly overestimated whereas lower values are neglected (Legates and McCabe, 1999). For the quantification of runoff predictions this leads to an overestimation of the model performance during peak flows and an underestimation during low flow conditions. Similar to r2, the Nash-Sutcliffe is not very sensitive to sys- tematic model over- or underprediction especially during low flow periods.

$\endgroup$
2

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.