Dynamic Outlier Detection in loop with Random Forest regression in R

Question

I'm using randomforest regressor to predict values. I'm trying to make a automatic learning model. My database has let's say 100 rows. So, I train my model with only 10 rows. And one by one I want to predict a value and see if with only 10 rows, my random Forest can predict the value correctly.

If not, then I will insert this new row in the trainset (with 10 values, now becoming 11) then try again with the next value and so forth until 100.

Is there a way to find out, before predicting the value with the model, if randomForest can precisely predict the value? Would an outlier detector be useful?

I am looking for something that will say : Oh I have never seen a value like this, I can't predict it precisely.

I tried using the lofactor function from DMwR package but the outlier method just finds the outlier of the whole TrainSet and not wether or not the last value I added in the Trainset is an outlier or not.

I hope this is somewhat clear, thank you all

Answer 1

Is there a way to find out, before predicting the value with the model, if randomForest can precisely predict the value?

Well, if the values of the predictors (especially the important ones) for the new data point(s) are outside the range of values that the model was trained on, it is much less likely to generate accurate predictions. This is because RFs do not extrapolate well, essentially predicting that the response beyond the most extreme points used for training is completely flat. But besides this, I do not see a way to generate such an expectation without actually using the model to make the prediction.