I was trying myself in kaggle CIFAR competition, I trained lots of classifiers but get the same result/fail (don't know how to treat them), maybe someone could help me figure what i'm doing wrong.

  1. So in the beginning I converted images to vector (included all colours) and tried to train classifier on a sample of training set. There was no preprocessing and I used random forest method. It scored about 70% percent on a training set, however when I try to apply my classifier on test set it classifies all points as one class. (in sample size was about 10k points in 3k dimensions with 500,1000 trees)
  2. Then I figured that I have to do dimensionality reduction, I figured out the amount of features I'm needed (nFactors library in R) and tried to train random forest classifier again on ICA projections. Prediction on test set again spits out only one class.
  3. Next thing I figured that maybe my method is not calibrated enough, so I sampled small amount of data and tuned svm classfier on it. Then I trained it on test sample, which resulted in 80% error on training set, but when I predict class on test set I still get one class prediction. Then I also tried create LogitBoost classifier, it classifies 50% of data quite correctly (leaving rest 50% NA's) but when i try it on test sample it results in all NA's.

What am I doing wrong? It looks like my classifier can't catch any underlying structure, though I tuned parameters, increased amount of trees to like 5k and did feature extraction, but still I get only one class prediction. (usually the one that have biggest error) Since other teams are actually doing quite well it means that something gone really wrong.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.