How can you convert sum of squares deviation to an r-squared value?

Question

I would like to calculate the r-squared value for some regressions. The model (its in a GUI) I am using gives me "goodness of fit" in terms of the sum of squares deviation. I am using Ecopath with Ecosim 6.6.5. The weight is the value in ( ) for the observed data.

This goodness of fit measure is a weighted sum of squared deviations (SS) of log biomasses from log predicted biomasses.

How can I derive the total sum of squares? How can I convert the SS value to an r-squared value?

For example, in the figure below, the SS value is outside the ( ). The model is the line and the points are "observations".

Answer 1

According to Section 9.2 of a User Guide for "Ecopath with Ecosim," what you are showing seem to be fits of a dynamic coupled differential-equation model to time-series data of the biomass of (potentially multiple) predator and prey species. The quote in the OP explaining the SS values (on page 81 of the User Guide) is followed by an explanation of the weights:

Each reference data series can be assigned a relative weight using a simple spreadsheet in the search interface, representing a prior assessment by the user about relatively how variable or reliable that type of data is compared to the other reference time series.

So this isn't the same as an ordinary (or weighted, in the usual sense) linear least-squares regression. There is a dynamic model fit to multiple outcomes observed over time, with outcomes contributing differentially to the fit of the model depending on their estimated reliability. The time-series aspect of the modeling adds additional complications.

The simplest thing would be to proceed as @Dave suggests in another answer. For each type of biomass, you could use the weighted residual sum of squares as the numerator of the fraction shown on the right, provided that you similarly weighted the sum of squared deviations of the individual log-biomass values from the mean log-biomass over the observations that form the denominator. That should be easy to calculate from the data you provided to the model.

There are problems with that.

First, even with a simple ordinary least squares model, you can get an arbitrarily high $R^2$ just by fitting a more complex model. Thus an $R^2$ adjusted for the number of fitted parameters is typically preferred. Yet this model is fitting a very large number of parameters to handle the interactions among the species in the ecosystem.

Second, time-series data can be lead to anomalously high $R^2$ values due to autocorrelation among observations. As the User Guide says on page 94, you need "to account for autocorrelation in the model residuals that is expected even under the null hypothesis."

Section 9.5.7 of the User Guide, "Time series random effects," indicates that there might be a way to get a global F-statistic "under the null hypothesis that all of the deviations between model and predicted abundances are due to chance alone." That uses Monte Carlo simulations to deal with the autocorrelation problem. That would seem to be a more useful estimate of model performance than $R^2$ values for individual (log) biomass values. I don't have any experience with this software, however, so I suggest that you work closely with someone who does to make sure that I'm interpreting that section of the User Guide properly.

Answer 2

In simple settings, there are a number of equivalent ways of defining $R^2$. In less straightforward settings, the ways I would calculate is according to the following.

$$ R^2=1-\left(\dfrac{ \overset{N}{\underset{i=1}{\sum}}\left( y_i-\hat y_i \right)^2 }{ \overset{N}{\underset{i=1}{\sum}}\left( y_i-\bar y \right)^2 }\right) $$

The top of the fraction is the sum of squared deviations. The $y_i$ are the observed values, and the $\hat y_i$ are the corresponding predicted values. ($N$ is the sample size.)

The bottom of the fraction looks similar but uses $\bar y$ instead of $\hat y_i$. This means that, instead of subtracting the predicted value, you subtract the overall mean of $y$, regardless of true or predicted value.

If this sounds like the beginning of a variance calculation, you are correct. If you have the variance, you can calculate the denominator by multiplying by $N$ or $N-1$, depending on how the variance was calculated. If you do not know, I would assume (while conceding there is a chance this is incorrect) the calculation to have divided by $N-1$, but, at least for a large sample size, the difference should not be much. If you have the original data, then you have the ability to calculate the denominator on your own without caring about any reported variance and how that variance was calculated.

Once you have the numerator and denominator, just plug those values into a calculator to get your $R^2$.

Since you take the $\log$ of your $y$, you would do all of this on the logarithms, not on the original values.