I was working on the scikit-learn-london dataset (on kaggle) and I was trying to do a binary classification task. The dataset is shortly described below:

  • 40 features (all numerical, not categorical)
  • 1000 training examples
  • 9000 test examples
  • labels 0 or 1

What I tried

I tried a bunch of classifiers and got approximately 90% cross validation accuracy. I used feature scaling, normalization, dimensionality reduction (with pca), feature selection and so on. The accuracy did not improve.

I also tried neural networks from 1 to 10 hidden layers, from very small to very large number of neurons. I used dropout and l2 regularizations. The accuracy did not improve.

If I'm not mistaken, theoretically, neural networks can fit every dataset with a good choice of parameters. So, I was about to conclude that this is the best accuracy one can get with this dataset.

What others did

Then I saw some people doing the following: Merge train and test datasets. Replace each row with gaussian mixture probabilities (without any extra preprocessing). These new features give around 99% accuracy with almost any classifier.

My question

  1. How could these people foresee that replacing rows with gaussian mixture probabilities would work?
  2. Is this a standard step in data preprocessing, or somewhat revolutionary/brilliant idea?
  3. Merging train and test sets (only rows, not labels) seems a little bit of cheating to me, isn't it?
  4. Can I ever find a neural network architecture and hyperparameters that works just right for this problem? (without gaussian mixture)

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.