I have 199 data, 77-diagnosed and 122-healthy. Each data has 12076 feature (gene expression). When I split data into train and test (N/2 for train and N/2 for test) and run feature selection on train-data and select best features and then run the same feature selection on all the data and select features, the selected features in two situation are so different(for example in the train data 259 features are selected and in the all data 119, and the number of common features is 28). The difference between the two categories is not impressive, and I don't know how to separate them. I tried information gain, random forest, chi-squared, sub-lasso and cfs (from FSelector package in R), but none of them works out. I need to reduce the features and prevent over-fitting and use a classifier to discriminate two classes, I tried SVM as classifier and also KNN (I know it is for clustering but all the results are desperating so I just tried it). The PC1-PC2 plot of data is also very bad (data points are not separable at all). When I divide people in dataset based on the age and sex, the PC1-PC2 plot is very good, but the results remains disappointing (for example when I choose male under 9, the data size reduces to 41 and I think that's the problem). The accuracy after classification is about 65 - 70%. Can anyone guide me how to make it better?