Is the convolutional neural network always the best model for image classification? Are there any situations in image classification problems, when the convolutional neural networks do not achieve the state of the art results? From what I have seen so far it appears that some kind of CNN always outperforms other models. 
 A: 
Are there any situations in image classification problems, when the convolutional neural networks do not achieve the state of the art results? 

If the training set is small, the traditional SIFT/HOG/... features may yield better predictions than a CNN.
A: There was recently published a paper in Science a Bayesian Program Learning classifier that outperformed the most recent deep learning approached on letter classification.  The model was even able to determine when it had been provided letters from a new language based on a single sample, and then identify subsequent letters that came from the same language based on that one letter.  It showed human levels of generalization in a "visual Turing test".  Whereas deep learning needs giant data sets to perform nearly all tasks, this model was able to to work more like a human being (ie. we don't need to read volumes of books on Chinese to realize we aren't reading English).
Also, while neural networks can do amazing things, they generally take a lot of skill to tune them and get them to work.  Even for professional data scientists, unless they specialize in neural networks, they will often get better results with traditional models.
