By myself when you are approaching to a new data set you should start to watch to the whole problem. First of all get a distribution for categorical features and mean and standard deviations for each continuous feature. Then:
- Delete features with more than X% missing values;
- Delete categorical features when a particular value gets more then 90-95% of relative frequency;
- Delete continuous features with CV=std/mean<0.1;
- Get a parameter ranking, eg ANOVA for continuous and Chi-square for categorical;
- Get a significant subset of features;
Then I usually split the classification techniques in 2 sets: white box and black box technique. If you need to know 'how the classifier works' you should choose in the first set, eg Decision-Trees or Rules-based classifiers.
If you need to classify new records without building a model should should take a look to eager learner, eg KNN.
After that I think is better to have a threshold between accuracy and speed: Neural Network are a bit slower than SVM.
This is my top five classification technique:
- Decision Tree;
- Rule-based classifiers;
- SMO (SVM);
- Naive Bayes;
- Neural Networks.