I haven't found any literature on the application of Random Forests to MNIST, CIFAR, STL-10, etc. so I thought I'd try them with the permutation-invariant MNIST myself.
In R, I tried:
randomForest(train$x, factor(train$y), test$x, factor(test$y), ntree=500)
This ran for 2 hours and got a 2.8% test error.
I also tried scikit-learn, with
RandomForestClassifier(n_estimators=2000, max_features="auto", max_depth=None)
After 70 minutes, I got a 2.9% test error, but with n_estimators=200 instead, I got a 2.8% test error after just 7 minutes.
With OpenCV, I tried
rf.train(images.reshape(-1, 28**2), cv2.CV_ROW_SAMPLE, labels.astype('int'))
This ran for 6.5 minutes, and using
rf for prediction gave a test error of 15%. I don't know how many trees it trained, as their Python binding for Random Forests seems to ignore the
params argument, at least in version 2.3.1. I also couldn't figure out how to make it clear to OpenCV that I want to solve a classification problem, rather than regression -- I have my doubts, because replacing
astype('float32') gives the same result.
In neural networks, for the permutation-invariant MNIST benchmark, the state of the art is 0.8% test error, although training would probably take more than 2 hours on one CPU.
Is it possible to do much better than the 2.8% test error on MNIST using Random Forests? I thought that the general consensus was that Random Forests are usually at least as good as kernel SVMs, which I believe can get a 1.4% test error.