Using $R^2$ to evaluate out-of-sample performance In this paper the $R^2$ is used to evaluate out-of-sample predictions for several methods including neural networks and tree based methods (see section 3.3 Evaluation and Validation). 
How is the out-of-sample $R^2$ computed and is it a valid performance measure? 
Edit: The link did not work. I fixed it now.  
 A: The coefficient of determination, $R^2$, value can be calculated as 
$$R^2 = 1 - SS_{res}/SS_{tot}$$ where the total sum of squares is
$$SS_{tot} = \sum_{i}(y_i - \bar{y})^2$$ 
and the residual sum of squares is
$$SS_{res} = \sum_{i}(y_i-\hat{y_i})^2 = \sum_{i}e_i^2$$ 
To apply the above equations to out-of-sample predictions you could use $y_i$ and mean $\bar{y}$ from your test data. This seems like the most obvious way of calculating out-of-sample $R^2$.
If the model prediction is better than simply assuming a constant fit equal to the mean, then the $R^2$ will be greater than zero. As $SS_{res}$ goes to zero, the $R^2$ would approach one. Not that mean squared prediction error (MSPE) would also go to zero as $SS_{res}$ goes to zero. Therefore, if there is low MSPE we would expect high values of $R^2$. The $R^2$ value can also be interpreted as one minus the ratio of MSPE to variance.
$$ R^2 = 1 - MSPE/Var(y) = 1 - \frac{\frac{1}{N}\sum_{i}(y_i-\hat{y_i})^2}{\frac{1}{N}\sum_{i}(y_i-\bar{y})^2} $$
One of the main criticisms of $R^2$, that would apply to out-of-sample as well, is that if the data is very noisy the $R^2$ may be low even if the model fits the data well. Also, the $R^2$ could be high even if the functional form of the model is different than that of process that generated the data (e.g., a linear fit to a quadratic function).
