I have a classification dataset of ~100,000 rows and ~200 features. Within the dataset my predictor variable (Y) is an integer value between 0-55, therefore I am trying to predict 1 of 56 possible classes. I split my training/testing set into 80/20% and performed an extensive 10-fold cross validation exercise to tune the parameters and fit the final model. I end up a very high accuracy (and F1) score in the training set (~90%) but a very slow accuracy (and F1) score in the testing set (~10%).
EDIT 1: The training and testing set are split randomly. About 150 of the features are binary features (0 or 1) and the others are continuous values which I center and scale to be between 0 and 1.
I have tried numerous learning algorithms (SVM, NN, logistic regression, PCA + SVM) and believe that the 10-fold CV should have eliminated overfitting as best as possible. However, nothing that I try seems to yield any meaningfully different results.
Can anyone suggest new ways to increase accuracy of the testing set?
Caveats:
1) This is real world data so getting more of it is very expensive and time consuming.
2) We need these 56 classes so cannot simply eliminate any.
Thanks and any suggestions are appreciated.