How to use residuals to train a model? I've read in a couple places about people using residuals from predictions of other models to improve the model's accuracy.  Anyone have a link about how to carry this out in R, ideally with some example code?
UPDATE:  This is in the context of reading the papers about the Netflix prize where k-nearest neighbors (KNN) models were trained using residuals from other KNN models.
 A: This technique is used in a variety of ways. For example, boosting. There are also various multi-level classifications algorithms that do similar things. The idea is to see where your first level makes mistakes and concentrate further learning in that area, keeping what you have and building on it rather than changing what you have.
A: What kind of model you are going to train? The residuals is different with frequentist and bayesian statistics. In frequentist statistics, residuals is a kind of MSE (Mean squared error) or risk function. Frequently, you would like to optimize the risk function $\mathbb{E}_\theta\left(L(\theta,\delta(X))\right)$, $L$ is a loss function, $\delta$ is your expected value.
You talk model's accuracy. First you should give a mathematical definition for your specific model's accuracy like $L(\theta,\hat{\theta})=\left(\theta-\hat{\theta}\right)^2$ or some loss function is proper for your model.
A: I ended up just taking the residuals as y and the original independent variables and training a randomForest on the residuals.  Worked ok.
