How can I improve my classification model(s), when it gives training, cross validation and test accuracies all close to 68%-69% only? I'm performing a binary classification with both logistic regression and SVM, where I've 80%-20% train-test split of the 10000 samples, each with 11 features. In my problem, the features are the data related to a car application, e.g. the time a ride request was made, the time the corresponding booking request(s) was/were made, the latitudes and lognitudes of the rider and the driver etc. The response variables are driver's response 1 (ride request accepted) or 0(ride request not accepted). After classification, I'm getting around 68% accuracies for all of training, cross validation and classifcation, and out of the response {0,1}, about 64% are 1 and 36% is 0. When I perform a grid search to find the best parameters for both linear and kernel SVM, I still get around 69% accuracies.I also tried logistic regression and Naïve Bayes, but the accuracy range in all cases were between 65%-69%.
From a high level, my question is: what should I do next to find a better model?
For example, 1) should I worry about class imbalance, since the classes are 64%-36% distributed? 2) a. since the number of features (11) is much less than that of samples (10000), do I still need to do PCA to improve accuracy? b. Should I do PCA in any case to get better understanding of the 2D projected data? 3) I'm not sure if there's lot of noise, as the data features are the car data as described; but if there's noise, how can I identify them?
Any help will be highly appreciated?
 A: Feature selection is one heck of a job and it is a huge domain. Well, in case of binary classification using SVM (or any other method as a matter of fact), the approach I choose is the use of Probability Density Functions (or PDFs). Let me put out a picture:

 Why do I use PDFs?
Ok. So, the goal of SVM in your case is to classify your data into two classes. How does SVM work? It aims to find a plane to divide your data into two classes (a line in 2D, a plane in 3D and a hyper-plane in n-D). 
When you trace a normalised PDF of a known data set for the two classes, the two curves tend to either be completely separated, maybe overlap a little or completely overlap. Here are three examples of a data set that I use. These are some examples from ECG signals. I have traced the mean of the data (Plot.1), the mean absolute deviation (Plot.2) and a slope Q-R (Plot.3) from ECG signals.

Plot.1 

Plot.2 

Plot.3 
As you can see in Plot.1, the red and blue curves on the right side almost completely overlap. In Plot.2, they are almost separated while in Plot.3, they are totally separated. 

How is this useful?
The dashed line in the three plots above, that is very similar to the SVM margin for classification for data both in feature space and the hyper-space (if you're using kernel functions). So if the separating hyper-plane is not able to separate the data into two classes, no matter what kernel or any feature set you use, the learning step tends to fail resulting in low accuracies (That will in turn fail your testing phase). If you have a known data set, you could trace such PDFs and see how the features appear. And if you're able to find say 5-6 features out of 11 that you have get good separation (as in Plots 2 and 3 above), I guarantee you that the learning step will be a 100% accurate and your testing step too will give you a very high accuracy. This is known as physical analysis of signal features (well, that's what they call it here at least :P).

Advantages of using this approach
As most work suggests, learning data should be at least 66% of the total data set. By using this approach, I was able to obtain very similar results even with learning data set size as low as 45% of the total data set.
PS: The PDF plots were traced using the moving average curve smoothing technique; you could use other techniques too.
