Can a very bad Coefficient of determination ($R^{2}$) not be indicative of model performance? Thanks in advance for the advice.
I am trying to build a generalized linear model that has many predictors.  The $R^{2}$ value of the model is quite low (.21), but when I use the model to predict against my validation set I am getting very good results.
I was under the impression that a low $R^{2}$ value generally means that the predictive power of a model is low.  What could be going on here (I am looking for reasons why a model may make good predictions but have a low coefficient of determination)?
My training and validation sets have a similar distribution and I believe my validation and training sets to representative of the whole space.
 A: I'm aware that the question was asked a long time ago but in case anyone stumbles across it in the future - An Introduction to Statistical Learning (Chapter 3.2) has a good explanation to that question:

The $R^2$ statistic has an interpretational advantage over the $RSE$, since unlike the $RSE$, it always lies between 0 and 1. However, it can
still be challenging to determine what is a good $R^2$ value, and in general, this will depend on the application. For instance, in certain problems in physics, we may know that the data truly comes from a linear model with a small residual error. In this case, we would expect to see an $R^2$ value that is extremely close to 1, and a substantially smaller $R^2$ value might indicate a serious problem with the experiment in which the data were generated. On the other hand, in typical applications in biology, psychology, marketing, and other domains, the linear model is at best an extremely rough approximation to the data, and residual errors due to other unmeasured factors are often very large. In this setting, we would expect only a very small proportion of the variance in the response to be explained by the predictor, and an $R^2$ value well below 0.1 might be more realistic!

A: Keep in mind what the equation for $R^2$ is.
$$
R^2 = 1 - \dfrac{
\sum_{i=1}^n \big(
y_i - \hat y_i
\big)^2
}{
\sum_{i=1}^n\big(
y_i - \bar y
\big)^2
}
$$
If you are measuring model performance in terms of the numerator (or something equivalent, like $MSE$ or $RMSE$), and you find that to be acceptable despite a low $R^2$, it means that the denominator is relatively large (in other words, much variation in the unconditional $y$). If the numerator performance is poor despite a high $R^2$, it means that the denominator is relatively small (little variation in the unconditional $y$).
It is reasonable to think that you have a low $MSE$ despite a low $R^2$ simply because you don't have that much variability in the unconditional $y$ to explain.
