I am new to machine learning and have been trying to learn. I have a drug ingredients data as independent variables and its efficacy value as the dependent variable.
I split it into the train and test set (0.8:0.2) and fit the model to the training set. Then I tried to use the random forest method to do regression.
The resulting model when used to predict the train set give $R^2$ accuracy of 0.97 and test set give 0.82. I have been trying to change the RF parameters but this is the highest test accuracy I could achieve.
I also find the OOB score of 0.85.
My question is, is this approach right? I have been researching about random forests on the internet and found the way random forests work is by doing bagging, therefore simulating CV. Is splitting the data the right thing to do? And can $R^2$ score be used, or OOB score is the one to go to know the model accuracy? Maybe both?