0
$\begingroup$

I was working on the scikit-learn-london dataset (on kaggle) and I was trying to do a binary classification task. The dataset is shortly described below:

  • 40 features (all numerical, not categorical)
  • 1000 training examples
  • 9000 test examples
  • labels 0 or 1

What I tried

I tried a bunch of classifiers and got approximately 90% cross validation accuracy. I used feature scaling, normalization, dimensionality reduction (with pca), feature selection and so on. The accuracy did not improve.

I also tried neural networks from 1 to 10 hidden layers, from very small to very large number of neurons. I used dropout and l2 regularizations. The accuracy did not improve.

If I'm not mistaken, theoretically, neural networks can fit every dataset with a good choice of parameters. So, I was about to conclude that this is the best accuracy one can get with this dataset.

What others did

Then I saw some people doing the following: Merge train and test datasets. Replace each row with gaussian mixture probabilities (without any extra preprocessing). These new features give around 99% accuracy with almost any classifier.

My question

  1. How could these people foresee that replacing rows with gaussian mixture probabilities would work?
  2. Is this a standard step in data preprocessing, or somewhat revolutionary/brilliant idea?
  3. Merging train and test sets (only rows, not labels) seems a little bit of cheating to me, isn't it?
  4. Can I ever find a neural network architecture and hyperparameters that works just right for this problem? (without gaussian mixture)
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.