1
$\begingroup$

I'm using RandomForest to predict some financial data (doing binary prediction). I have 4500 rows in my training data and 1500 in my test data (I'm actually using nested cross validation on a larger data set but let's keep it simple).

Feature Set 1 has 7 variables, and it delivers OOS 69% accuracy (baseline accuracy is 59%). Feature Set 2 has 36 variables, and it delivers 66% accuracy on same OOS data.

The two feature sets are very uncorrelated. When I put both sets together and train an RF model, the result is around 68% accuracy. I'm training 300 trees, and increasing trees on combined feature set doesn't improve results.

Why does this happen? Both feature sets seem to provide orthogonal predictive value - why does using them simultaneously not yield better performance than each on their own? Is there any solution?

EDIT: I have found that even though the variables in the two feature sets are quite different, the predictions generated by models trained on the two sets are pretty similar. I think this might be the reason why I'm seeing no improvement when combined?

$\endgroup$

1 Answer 1

0
$\begingroup$

As I mentioned in my EDIT, I think I have found the answer. Even though the features in the two sets seem very different, the predictions generated by the two sets independently are very similar - so combining them didn't improve the accuracy.

$\endgroup$
1
  • $\begingroup$ Your answer could be improved with additional supporting information. Please edit to add further details, such as citations or documentation, so that others can confirm that your answer is correct. You can find more information on how to write good answers in the help center. $\endgroup$
    – Community Bot
    Jan 4, 2022 at 6:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.