8
$\begingroup$

I am quite new to digit recognition, and I have noticed that many tutorials use the SVM classification, for example:

I would like to know if there are any (domain specific) benefits for that tool, compared to e.g.

  • Deep learning neural networks
  • Classification based on k-means

Thank you for any comment. Clarification, why SVM is a wrong tool for digit recognition, will be appreciated, too.

$\endgroup$
9
$\begingroup$

Tutorials for many different classifiers are based on digit recognition, particularly the MNIST data set. I think this is largely because this dataset is readily available, easy to understand, and requires minimal preprocessing. It's also a common benchmark dataset in the literature.

I'm not aware of arguments that any particular family of classifiers is intrinsically most suitable for digit recognition. I think this would be a difficult argument to make because 1) The distribution of digits comes from the real world and we don't have a comprehensive mathematical description of it, and 2) The set of all SVMs, all neural nets, etc. is infinite, and we've only managed to explore a tiny subset. Perhaps the best known neural net is better than the best known SVM, but a new kernel function is lurking somewhere in that vast, undiscovered space that would make SVMs better. Perhaps there's another neural net architecture that would be better still. I suppose arguments could be made in certain cases (e.g. naive Bayes on raw pixels is probably a bad idea because its assumptions are blatantly violated).

There are many reasons to prefer each type of classifier over others in different circumstances (e.g. time/memory required for training/evaluation, amount of tweaking/exploration required to get a decent working model, etc.). These reasons aren't specific to digit recognition, so I won't go into them here.

There are certainly domain-specific tricks than can make classifiers more suitable for digit recognition. Some of these tricks work by increasing invariance to particular transformations that one would expect in handwritten digits (e.g. translation, rotation, scaling, deformation). For example, the digit '0' should mean the same thing, even if it's shifted to the left and warped a little bit. Some of the tricks are specific to the family of classifiers. For example, this kind of invariance can be had using certain SVM kernels, spatial transformer layers in neural nets, or probably an invariant distance metric for K nearest neighbors. Other tricks can be used with many classifiers. For example, the dataset can be augmented with many transformed copies of the digits, which can help the classifier learn the proper invariance.

SVMs, neural nets, and even kNN have achieved good performance on the MNIST dataset. Other methods work too. The best performance I've personally heard of has been with neural nets. Some results are accumulated across different papers here and here.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.