# Sampling to get around randomForest's 32 factor level limit

I'm trying to work around the randomForest package limit of 32 levels for factors.

I have a data set with 100 factors in one of the variables.

I wrote the following code to see what things would look like using sampling WITH replacement and how many tries it would take to get certain % of factors selected.

sampAll <- c()
nums1 <- seq(1,102,1)
for(i in 1:20){
samp1 <- sample(nums1, 32)
sampAll <- unique(cbind(sampAll, samp1))
outSamp1 <- nums1[-(sampAll[,1:ncol(sampAll)])]
print(paste(i, " | Remaining: ",length(outSamp1)/102,sep=""))
flush.console()
}

[1] "1 | Remaining: 0.686274509803922"
[1] "2 | Remaining: 0.490196078431373"
[1] "3 | Remaining: 0.333333333333333"
[1] "4 | Remaining: 0.254901960784314"
[1] "5 | Remaining: 0.215686274509804"
[1] "6 | Remaining: 0.147058823529412"
[1] "7 | Remaining: 0.117647058823529"
[1] "8 | Remaining: 0.0980392156862745"
[1] "9 | Remaining: 0.0784313725490196"
[1] "10 | Remaining: 0.0784313725490196"
[1] "11 | Remaining: 0.0490196078431373"
[1] "12 | Remaining: 0.0294117647058824"
[1] "13 | Remaining: 0.0196078431372549"
[1] "14 | Remaining: 0.00980392156862745"
[1] "15 | Remaining: 0.00980392156862745"
[1] "16 | Remaining: 0.00980392156862745"
[1] "17 | Remaining: 0.00980392156862745"
[1] "18 | Remaining: 0"
[1] "19 | Remaining: 0"
[1] "20 | Remaining: 0"


What I'm debating is whether to sample with or without replacement.

1) getting a sample of 32 of the 100 factors,
2) using those lines to run the randomForest,
3) predicting the test set with the randomForest and
4) repeating this process either
a) 3(WITHOUT replacement) or
b) 10-15 times (WITH replacement).
5) taking the 3 or 10-15 predicted values, finding the average and using that as a final predictor.


I'm curious if anyone has tried something like this or if I'm breaking any rules (introducing bias, etc.) or if anyone has any suggestions.

NOTE: I've cross-posted this question on Stack-Overflow.

-
It's not immediately obvious to me which site it should be on, but generally you should at least note that you've cross posted it at Stackoverflow as well. –  joran Jan 8 '12 at 0:42
Thanks, will do. –  screechOwl Jan 8 '12 at 0:54
Do not cross-post the same question on multiple sites, it is not permitted. The question on Stack Overflow is now closed. This is the more appropriate site as this is not about programming but asking about sampling without replacement. –  casperOne Jan 8 '12 at 17:12

You may also repeat this few times (with different levels to 0:99 mappings) and this way emulate your intended random scan loop within one model training, only on information system with $2\text{few}-1$ more attributes.