How to validate if factors are significant or not? Or how to choose factors? Example in Logit, svm, Naive Bayes, and Random Forest? One sentence summary, what is a systematic factor selection method for any of  Logit, svm, Naive Bayes, or Random Forest models? 
Let's focus on four most common classification methods, logit regression, svm, naive bayes, and random forest. 
Can you please explain how to choose factor? How to determine factor significance? And how to do dimensional reduction? Hints, illustrations, or example on those models, or on your model will be appreciated!
To my knowledge, below are my thought:
stepwise, AIC, BIC, bootstrap, PCA, R^2 etc.. these can be used in factor analysis
I am not from a math/stats background, my knowledge about these are in a high degree of fragmentation. That is why I ask this question, wanted to know how professionals work, from a systematic method or systematic thinking. 
 A: The question is broad covering several general topics so I will give you partial answers.
When you have several candidate predictor variables there are several criteria that can be used for subset selection.  AIC and BIC are two of them that penalize for using too many parameters (this is to avoid overfitting). You can find several others discussed in many of the posts here. Most of these use prediction accuracy as the criterion.  R$^2$ is not good because it will never decrease when additional variables are added. The adjusted R$^2$ fixes that problem but is only a little better. PRESS and Mallow's $C_p$ are two other measures.
Frank Harrell's regression book discusses this and he has given his opinions here in a number of posts.
There are others that can be used based on other criteria such as Gunter's best for detecting qualitative interactions. This approach is useful in medical research, particularly clinical trials.
Principal Component Analysis (PCA) finds orthogonal components (linear combinations of variables) that maximize that variance explained in a multivariate data set. Using the first few principal components can explain most of the variability and can be viewed as dimensionality reduction.
The bootstrap can be used for a wide variety of statistical problems.
Some references: On the bootstrap
Bootstrap Methods: A Practitioners Guide 2nd Edition (2007) M. R. Chernick Wiley
Gunter's method and more
Statistical Methods for Dynamic Treatment Regimes: Reinforcement, Causal Inference and Personalized Medicine (2013) B. Chakraborty and E. E. M. Moodie Springer
Harrell's regression book
Regression Modleing Strategies with Applications to Linear Models, Logistic and Ordinal Regression and Survival Analysis 2nd Edition (2015) F. Harrell Springer
