I have a dataset with 5,818,446 lines and 51 columns, where 50 of them are predictors. My response is quantitative, so I am interested in a regression model. I am trying to fit a random forest to my data using the caret package. However, I do not have enough RAM to do it.
I've been looking for solutions to my problem. Besides having a more powerful computer, it seems I can make use of bagging to solve my problem. Therefore, my idea is as follows:
Create both train and test partitions from my original dataset
Sample with replacement a small part of my train dataset into R (let's say 1% of it, i.e., 58,185 lines)
Fit a random forest to this small part of data
Save the model result
Repeat steps 2-4 1,000 times
Combine these 1,000 models obtained from steps 2-5
However, random forest itself uses bagging to fit the model to the data and thus I am not sure if my approach is correct. Therefore, I have some questions for you:
i) Is my approach correct? I mean, since I do not have enough RAM in my system, is it correct to fit many different random forest models to different chunks of data and combine them after?
ii) Assuming my approach is correct, 1% of data is a good rule of thumb for my sample size? Even with 1% of data, I still have $n \gg p$.
iii) Assuming my approach is correct, is there a number of replications for models I should be using? I thought of 1,000 because of reasons.
l1
regularization, which typically drops insignificant components' weights to very nearly zero, so you can see by inspection which trees you should keep. $\endgroup$