1
$\begingroup$

Statistical models in general are not well suited for applications outside the training range.

Using a random forest binary classifier aimed at detecting an event, I've trained it using 100 data points due to data limitation. I then applied it in a similar dataset, which had 2000 data points (but no y values to train with) to predict the number of events. Despite having similar statistical properties such as mean and standard deviation, the larger datasets implies in more extreme values (tails). In fact, 5% of the larger dataset lies outside the training range.

That said, for this specific case of binary prediction, I reckon it could be ill-posed to rely on the RF model. However, does anybody have any suggestions or alternatives (parametric/linear models, tools, packages) on how to deal with cases like this?

Improve this question
$\endgroup$
2
  • $\begingroup$ Take a look at semi-supervised approaches: "few labeled - lots unlabeled" $\endgroup$
    – Laksan Nathan
    May 7, 2021 at 13:16
  • $\begingroup$ Thank you for the suggestion, I will take a look at it right away. I ran this sort of test in the data basically converting every exceeding value to the respective maximum or minimum value of the training set and the results did not change (it is a classifier based on thresholds after all). Would this be acceptable somehow? $\endgroup$
    – Henrique
    May 7, 2021 at 18:19

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.