First off, sorry if this a novice question! Relatively new to all this stuff.
Anyway, I'm working with 22 datasets that each have 180 observations of "Oddball" data and 720 observations of "Standard" data. I'm trying to use random forests for classification (i.e., oddball=1, standard=0). I understand there should be approximately equal trials/observations for both factors, but if I use 75% of the oddball data, then I'm barely using over 18% of the standard data. These data are pretty variable, and I think this could be problematic.
If I make four models, still using the same training data for each, am I overfitting my model? There's a lot more I've written, but this is basically what I'm trying to do:
jj = sample(1:180,(180*75),replace = F #Take 75% of all oddball data kk = sample(181:900,(720*.75),replace = F) #Take 75% of all standard data jj = sample(jj); kk = sample(kk) #Mix them up kk = matrix(kk,4) #Divide the standard data so there are 4 sets of equal numbers for jj samp1 = c(jj,kk[1,]) samp2 = c(jj,kk[2,]) samp3 = c(jj,kk[3,]) samp4 = c(jj,kk[4,])
I would then make four models (all while not touching the out-of-sample data) using each of these sample sets and average all their predictions to give me a "master" probability (i.e., an average of .8 would be deemed an oddball).
Is this overfitting the data? Is it even possible to overfit the data when using random forests? Is there something wrong with this intuition?
Thank you for anyone that helps! Your time and expertise is much appreciated.