Algorithm selection rationale (Random Forest vs Logistic Regression vs SVM) I want to understand the criteria of selection of ML algorithms i.e what are the guidelines on which algorithm to be selected in which case ?
The reasons I know are :


*

*Logistic regression to be picked in case we want to advise the impact on y variable on what change on any x variable.

*Random forest works good on mixed data and very effective for categorical data. Also it does feature selection first(so dimension reduction is not needed).

*Random forest not to be picked with high featured and multiple category data due to its high processing time.

*SVM works well with the closely placed data points like in image processing identification of dog vs cat.


But these are not sufficient enough to pick anyone, as i don't have any reason for why which algorithm not to be picked. 
Like when to choose SVM over Logistic regression or RF over Logistic regression.
The only rationale i have is the performance, so i run all algorithms and who ever performs best that i select(but this is not right way).  
 A: I think there is good news and bad news on this...
The bad news is that each and every choice of algorithm implicitly makes assumptions on the underlying probability distribution of the data (more precisely, it usually makes assumptions about the structure of $E[Y|X]$ where $Y$ is the target variable and $X$ is the variable responsible for producing the feature vectors).
Since we cannot actually say anything about it there is no other way than testing many different mode classes in order so select the best one. Even worse: there is a theorem (called no free lunch theorem) that tells us that for every model, we can generate a very weird dataset so that this mode becomes arbitrarily close to guessing, i.e. we can even mathematically prove that there is no such thing as a “magical mode selection device”, the only thing we can do is testing...
The good news is that in a “reasonable “ ML setup (<100 features, ~10000s of data rows) it does not really make a difference which model you use (as long as you optimize the hyperparameters and as long as it is not a “special” area like reinforcement learning or imaging or NLP, in these areas, NNs are successful). The reason I use to “explain” this to myself is that for each of the model classes you were talking about (except for linear and logistic regression) you can approximate any “not too bad” function (continuous on a compact interval) arbitrarily close. For forest like models this works as follows: there is a theorem that tells us that continuous functions in compact intervals are uniformly continuous and for those functions it is easy to see that they can be approximated arbitrarily close by step functions. Since you can build step functions with forests, forests can approximate continuous functions on compact intervals.
The only reason to exclude a model class (that comes to my mind) is -as you said- performance. If you deal with DNA like “rectangular” problems (millions of features, little amount of rows) then models based on decision trees will probably take forever. If it is millions of rows and few columns on the other hand, SVMs cannot deal with that so well.
Hope this helps...
