Bootstrapping estimates of out-of-sample error I know how to use bootstrap re-sampling to find confidence intervals for in-sample error or R2:
# Bootstrap 95% CI for R-Squared
library(boot)
# function to obtain R-Squared from the data 
rsq <- function(formula, data, indices) {
  d <- data[indices,] # allows boot to select sample 
  fit <- lm(formula, data=d)
  return(summary(fit)$r.square)
} 
# bootstrapping with 1000 replications 
results <- boot(data=mtcars, statistic=rsq, 
     R=1000, formula=mpg~wt+disp)

# view results
results 
plot(results)

# get 95% confidence interval 
boot.ci(results, type="bca")

But what if I want to estimate out-of-sample error (somewhat akin to cross-validation)? Could I fit a model to each boostrap sample, and then use that model to predict for each other bootstrap sample, and then average the RMSE of those predictions?
 A: The short answer, if I understand the questions, is "no".  Out of sample error is out of your sample and no bootstrapping or other analytical effort with your sample can calculate it.
In answer to your comment on whether the bootstrap can be used in checking a model with data  outside a training set:  two possible interpretations.
It would be fine, and absolutely standard, to fit a model on your training set with traditional methods and then use bootstrapping on the training set to check for things like distribution of your estimators, etc.  Then use your final model from that training set to test against the test set.
It would be possible to do a bootstrap-like procedure that involves a loop around:


*

*selecting a subset of the whole sample as your training set

*fit a model to that training set of the data

*compare that model to the testing set of the remaining data and generate some kind of test statistic that says how well the model from the training set goes against the test set.


And then considering the results of doing that many times.  Certainly, it would give you some insight into the robustness of your train/test process.  It would reassure you that the particular model you got was not just due to the chance of what ended up in the test set in your one split.
However, it's difficult to say exactly why but there seems to me to be a philosophical clash between the idea of a testing/training division and the bootstrap.  Perhaps if I didn't think of it as a bootstrap, but just a robustness test of the train/test process it would be ok...
A: Bootstrap is neither in-sample or out-of-sample test. 
Consider the bootstrap logic: 
1. a statistic is computed in the original sample;
2. a resample is constructed by sampling from the sample with replacement (this sample is considered to be a possible sample from the same population)
3. the same statistic a computed
4. step 2 and 3 are repeated and the distribution of the obtained statistics is then used to construct a confidence interval
Now translate this to the notion of out-of-sample testing, where you estimate a prediction model based on the original sample and then test out-of-sample. The out-of-sample sample should be any sample other than the original sample drawn from the same population. 
Resampling with replacement provides you with such a sample, or indeed many such samples should you so wish. Now you can use the original model estimates from your original prediction model to predict outcomes in the new resample(s). 
You can now compute a model-fit statistic to see if these predicted outcomes predict a similar share of the variation in the original sample and all of the resamples. Are all results comparatively similar, then overfitting is no issue. Are the results of resamples (significantly) worse than the model fit in the original sample, then you've got evidence of overfitting. 
When comparing different training models, you can select the model with the best (average) modelfit in the resamples. More advanced strategies involve the variance of the modelfit, but add little in my opinion.
Best wishes
A: This calls for the standard Efron-Gong "optimism" bootstrap.  In R you can do this:
require(rms)
# Allow age to interact with sex and age and BP to have nonlinear effects
# using restricted cubic splines (5 and 4 knots)
f <- ols(y ~ rcs(age,5)*sex + rcs(blood.pressure,4), x=TRUE, y=TRUE)
validate(f, B=300)

This will give you the bootstrap overfitting-corrected estimate of $R^2$, MSE, and other indexes.  To get a bootstrap overfitting-corrected calibration curve (estimate of relationship between $\hat{Y}$ and $Y$), run plot(calibrate(f, B=300)).
This type of bootstrap estimates the likely future performance of the final model on new subjects from the same "stream" of subjects.  Some observations are duplicated, triplicated, etc., and "training" and "test" datasets overlap during the bootstrap.  The bootstrap provides a highly competitive estimate of future performance, along the lines of 100 repeats of 10-fold cross-validation.
