What determines what classification algorithm you should use for a certain classification problem? e.g. If there is >5 features or you only have 1000 training examples, or there is multiple class's or it's binary classification use; logistic regression, SVM, ANN etc.
This flow chart is limited, but boils things down well for people new to the field. Beyond that, the dominating factors tend to be
- What's available in my libraries?
- What am I comfortable with?
- Which models have assumptions that match my data?
That last one is quite a deep question, and the ability to answer it effectively only comes with a lot of study. If you've got a good grounding in linear algebra and multivariable calculus, a general ML book like Machine Learning: A Probabilistic Perspective would be a good place to start.
It is pretty reasonable to start with a simple linear model and then work from there. Analysis of results will reveal the shortcomings of your features and/or algorithm. A significant advantage that you can get started quickly which is rather important in many practical applications.
The second step would be to start to use domain knowledge. Good analysis should result in a set of features that even a liner classifier might work satisfactory enough.