-1
$\begingroup$

In my understanding random forest model will keep one third of the data for testing the model. That means we do not need to explicitly test the model with new data.

Assume we build a random forest model with training data. After that we receive a new set of samples for each month.

Do we need to use the existing model for predicting the new set of samples or do we need to build an entirely new model with these samples?

$\endgroup$
0
$\begingroup$

A basic idea of machine learning is that you learn to generalize from one dataset to another. Thus, if you get a good cross-validation results on one dataset, you should be able to use the model again and again on new datasets.

That said, I would try to re-learn the model when new data arrives, automating the cross-validation process. Then, see if the model accuracy (on validation data) is increasing when adding new data. This will give you an answer: whether you need to re-learn the model when data accumulate or not.

$\endgroup$
0
$\begingroup$

Your first sentence seems to indicate that you conflate the model (in your case Random Forests) with a means of validating its performance on a given dataset (e.g. k-fold cross-validation). You should look at that, since for example choosing a different k in k-fold cross-validation might make sense depending on the data (see here).

To answer your question, not re-learning the model when new data arrives can lead to problems if

  • the original dataset was too small to learn your model well in the first place (an indication for this is if your cross-validation error on the original dataset is high)
  • the new data is qualitatively different from the original dataset. Was the data sampled in the same way or do you expect any bias in the new data?

The only case where I can imagine that not re-learning would make sense if you can exclude the two points above and re-learning has a high cost in time or money. If the cost is low I would re-learn anyway, just to be sure.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.