Random Forest cross-validation r2 is high but predictions on simulated data are bad I have a dataset with 46 million observations and 25 predictors.  I am training my model in the python h2o package like so:
import h2o

h2o.init(nthreads = -1, max_mem_size = "110g")
df = h2o.import_file(path = "C:/user/path_to_data.csv")

#predictors
x = [col for col in df.column if 'target' not in col]
y = 'target'

#establish model
m = h2o.estimators.H2ORandomForestEstimator(model_id="RF_defaults", nfolds = 5)

#train
m.train(x,y, df)

#view results
print (m.cross_validation_metrics_summary())

My result shows r2 values of about 0.93 across all 5 folds.
There is one variable in particular I am interested in seeing how predicted values compare to, called Year Since Fire.  To examine this I randomly selected 1,000 observations that were trained in the model, and then replicated each 70 while changing year of fire so that each ranged from 1 to 70, and kept all other variables the same as what they originally were.  In this way I then had 70,000 observations.  
I then used the model to predict (m.predict(new_data))the target variable through all 70,000 of these observations.  I then plotted the mean predictions for each Year Since Fire, and compare to the mean of the true observations in the original 1,000 pixels.
This is what the plot looks like (Albedo is the target variable):

and the r2 of this is only 0.13.
My main question is, if my cross validated r2 values or so high, why is the model predicting so poorly when I do this simulation exercise?
My initial thought was the variable Year Since Fire was not capturing most of the variation (it is number 5 in feature importance and explains 0.08 percent of the variation).  To test this I built 70 different models, one for each Year Since Fire and then applied the model appropriately, e.g. all pixels I wanted to predict on in Year Since Fire 1 were only modeled with a model built on Year Since Fire 1.  This is result is better, but still not great with an r2 of 0.6, and I am sure there is error being propagated by building 70 different models in the first place.

EDIT:
To provide information on how the data is sampled and simulated, 1 of the 1,000 sample pixels may look like this (without showing all 25 predictors):
Albedo  Year_Since_Fire  Temp  Prcp  DD  
0.6     50               20    100   300

This 1 observation is then replicated 70 times, while changing Year_Since_Fire:
 Year_Since_Fire  Temp  Prcp  DD  
 1                20    100   300
 2                20    100   300
 3                20    100   300
 4                20    100   300
 5                20    100   300
 6                20    100   300
 .................................
 70               20    100   300

This done for all 1,000 pixels, each with different initial values for the independent variables.
 A: For the question in your subject, I think the answer is that r2 is poor metric to be using. See http://data.library.virginia.edu/is-r-squared-useless/
I would instead use RMSE to judge the quality of your models. (Just preferring RMSE over MSE as it is in actual units, so easier to judge if "4.5" is a good enough model or not.) (I'd also be looking at MAE; if RMSE is much worse than MAE you could describe it as "When it gets it wrong, it really gets it wrong.")
Also, just to be aware, though I am sure you have enough data that this won't be a factor: the final model is built using all your data, not the 80% that each CV model uses. It is worth comparing the RMSE of each of the 5 CV models with the RMSE of the final model. If they are all close, then the random fold selection is reliable; if they are wildly different, you may need to try a different way of making the folds.
As for the rest of the question:

To examine this I randomly selected 1,000 observations that were trained in the model, and then replicated each 70 (times) while changing year of fire so that each ranged from 1 to 70, and kept all other variables the same as what they originally were

Unless I have have misunderstood, the behaviour you see makes sense for a tree algorithm, i.e. a non-linear algorithm. The "all the other variables" will send it down certain parts of the tree before it considers Year. If the falsified value for Year is something it has not seen before, it will be choosing the value nearest to what it has seen.
As an extreme example, if you have trained with:
a   b    answer
3   5    8
2   3    5

Then you change the b to be all 3s, it will give this result if a tends to come before b in the trees:
a   b    prediction
3   3    8
2   3    5

And this result if b tends to come first:
a   b    prediction
3   3    5
2   3    5

