Machine learning cookbook / reference card / cheatsheet?
There are numerous machine-learning approaches out there. Also there are numerous ways how to optimize their parameters and numerous ways to preprocess your data beforehand. It seems to be a common practice to initially implement some off-the-shelf approach (SVM, random forest) and then trying to beat the accuracy with more sophisticated approaches. I would like to know if there are any guidelines, recommendations or rules of thumb which approach to use in which context? Are recommendations data-specific or domain-specific (e.g. genetics, econometrics)? Also I would like to know how to account this multitude when writing results up. Trying out multiple approaches and only reporting the best one in your publication cannot be valid, right?