Classification problem with statistically insignificant variables I am working on a binary classification problem taking one categorical and four numeric variables. I started with t-test and logistic regression, which resulted in high p-values for all the variables I considered. 
As, all the variables I considered are statistically insignificant, what should be my next approach for the classification task?
 A: This only means that there are no linear relationships between predictors and the decision; there is still a chance some more complex method would find them (obviously at a greater risk of overfitting).
You may give it a try with random forest, it finds much more complex iterations than regression and still is pretty hard to overfit.
A: To summarize previous two posts: 


*

*You can either try to find new predictors which have higher predicting power. This is called feature engineering (http://machinelearningmastery.com/discover-feature-engineering-how-to-engineer-features-and-how-to-get-good-at-it/). 

*You can try and use a more complicated (non-linear) classifier. An overview in which you can try to select the right one specific to your demands, is given in the book the Elements of Statistical Learning or in this presentation: http://web.engr.oregonstate.edu/~tgd/classes/534/slides/part2.pdf. 
A: If your predictors are not significant, that generally means they are not useful for predicting your variable of interest. 
So you need to find new predictors, preferably some that are strongly related to your variable of interest. 
