Is a SVM (+Boost) faster than a NN to train with similar accuracy?

Question

I might possibly misunderstand something here. So please do tell me if I do.

At the moment, I am doing some research regarding the use of machine learning to detect a certain object.

Currently I am trying to figure what is better to use in my situation: a (Convolutional) Neural Network or Support Vector Machine (possibly boosted). I know there is also the possibility of Haar Cascade (using Adaboost). But I am just generally more interested in SVM and NN.

Hypothetically speaking: If I have about 400 images of a grey scale with the same resolution from an object which I like to detect with different lighting conditions, rotations and distance (but same resolution). Would training a SVM instead of a (possibly pre-trained) NN be more accurate and/or faster?

I know it really depends on the data-set, but I am trying to figure out if there is any way of telling before actually testing/implementing one of these methods. All the answers that I have found up to this point all state: It depends, but what does it depend on? Is it possible to, in layman's terms, to explain what could influence the choice regarding this situation?

Also, I wouldn't say that I am stupid, but I am not educated on a Uni level. So, although I wouldn't mind trying to read a research report on this topic, I highly doubt that I will understand most of whats normally written, especially when it comes to mathematical explanations.

I am planning to use already existing libraries such as TensorFlow, Caffe or just OpenCV/LibSVM (I know OpenCV uses an implementation derived from LibSVM) for this situation, based on what I could find on the internet. I am however a complete beginner regarding Machine Learning.

Answer 1

I will start from the very end of your question:

I am however a complete beginner regarding Machine Learning.

The problem you are trying to solve is not a beginner problem. Take it easy and start with beginner problems. There are many good learning resources for beginners. For example, the book The Master Algorithm by Pedro Domingos is generally praised for its readability and explaining ML in simple terms. There are online courses, and beginner guides. Following any of these is the good way to move from being a complete beginner to being a confident user of ML, even without academic education.

---

Regarding the actual question,

Would training a SVM instead of a (possibly pre-trained) NN be more accurate and/or faster?

You ask for two different things.

1. Which will be more accurate?

   Probably neural network. Neural networks are state-of-the-art at object detection. See also Neural networks vs support vector machines: are the second definitely superior?

2. Which will be faster?

   Probably SVM. Neural networks are notorious for their high computational requirements. See also Computational Complexity of Prediction using SVM and NN?

I say "probably" because, as you have already discovered,

All the answers that I have found up to this point all state: It depends, but what does it depend on?

it really depends.

• Performance (both in terms of speed and prediction accuracy) of neural networks depends entirely on their architecture. You can have a small neural network which runs fast but gives poor predictions. You can have a huge network which takes several days to train and it gives excellent predictions.

• Difficulty of the task depends on your data. Perhaps it is trivial to detect the objects in your images and perhaps it is not. The only way to see which is better is to try both and compare.

• Neural networks have many parameters and need many training samples to learn well. My guess is that 400 images may be a bit too little, but again, if the task is easy and you use a pre-trained network, it may work.

• ...