Is it possible to achive low error on MNIST using Random Ferns?

Question

I'm new in machine learning and i want to study how to use random ferns.

I read this paper Fast Keypoint Recognition in Ten Lines of Code and implement simple version of algorithm.

But then I tried to run this algorithm on MNIST dataset (without any pre-processing) I achieved only 7.5% error rate. I created 40 ferns and 40 pairs of points for each fern. I chose random subset of training data to train each fern (it improves error rate a little)

I suppose that 7.5% is not really bad result. But i was able to achieve 3% by simple Random Forest (CART)

So my question is: How i can improve this error rate and is 7.5% ok for Random Ferns? I do not want to obtain cool digits classifier but i want to know is my implementation works well and how to improve Random Ferns.

Answer 1

Take a look at discriminative ferns ensembles. Basically they train the classifier by minimizing a Hinge loss similar to that minimized by SVM's.

This way they improve the accuracy of classical ferns for the MNIST database.