I have a training dataset with a binary response variable, 6 independent variables, and 21,000 observations.
I've fit both an ordinary regression tree and a random forest (mtry = 2, ntree = 2000) and there is almost no difference between the two when each model is validated, using RMSE and predicted to actual ratio as goodness of fit metrics.
Is this to be expected with a small number of independent variables, or am I not using the right metrics to measure goodness of fit?