I am new to machine learning and have been trying to learn. I have a drug ingredients data as independent variables and its efficacy value as the dependent variable.
I split it into the train and test set (0.8:0.2) and fit the model to the training set. Then I tried to use the random forest method to do regression.
The resulting model when used to predict the train set give r2 accuracy of 0.97 and test set give 0.82. I have been trying to change the RF parameters but this is the highest test accuracy I could achieve.
I also find the OOB score of 0.85.
My question is, is this approach right? I have been researching about Randomforest in the internet and found the way Randomforest work is by doing bagging, therefore simulating CV. Is splitting the data the right thing to do? And can r2 score be used, or OOB score is the one to go to know the model accuracy? Maybe both?