When reading about image classification, the only occurring terms are "neural networks", "deep learning" and "CNN". It seems like there are no other methods for this task. I have worked with neural networks and see why it works so well, but I am just curious whether there are other methods used for this task as well?
Are there any other (efficient) methods for image classification besides using neural networks?
Before CNNs took off, people would use the following approach:
Multitude of feature extractors exists, such as SIFT or HOG. Extracted features can be further used to represent the image as a bag of visual words, building a histogram of features describing the image globally. You can move on and design intricate schemes how to incorporate spatial relations into this vector representation, only to find that what works well for one image domain fails miserably in another one. The result is a pipeline of handcrafted algorithms and a little change in any of its blocks may yield unpredictable changes in the output.
Looking at the overall scheme, you can see that it is the same what convolutional neural networks do: A series of convolutional and pooling layers is used to extract visual features which are later classified using a multilayer perceptron (fully connected layers).
The key of CNN's success is that the feature extraction part is not fixed. It provides a framework which is flexible enough to learn a suitable feature representation for the given task, yet constrained enough to be tractable and avoid overfitting. The classifier part is generally convenient since it allows joint training via backpropagation, but having the learned feature extraction part of the network, you could also use any other classifier such as SVM.
In this regard, neural networks are not a revolution shattering all the previous work, they are a natural evolution of a field with a long history.
This page seems to be a good source of lectures on the topic: http://people.csail.mit.edu/torralba/shortCourseRLOC/
