If you train a lot of machine learning algorithms on a problem, svm, nn, rf,...via i.e. caret, and I mean a lot of it,like hundreds or thousands (not that hard,considering all parameter tuning in validation), eventually you will find one that will work.
But that is data snooping. As you have trained so much models, I think that you have to test the hypothesis that you had been lucky, maybe with White Reallity Check test or Hansen Superior Predictive Ability test.
Around 99.9% of the papers, articles, posts,.... didn't use White's or Hansen's tests (0.1% are papers regarding stock trading via ML). I suppose this is because, on a normal basis, we use to train a few models(really?).
The question is:
Do you have any idea about the number of models required to consider data snooping terrible effects?
This is, if i'm choosing between 3 models, I think that the possiblity of getting good results by chance is low. But in choosing between 30? 300? 3000?