I was trying myself in kaggle CIFAR competition, I trained lots of classifiers but get the same result/fail (don't know how to treat them), maybe someone could help me figure what i'm doing wrong.
- So in the beginning I converted images to vector (included all colours) and tried to train classifier on a sample of training set. There was no preprocessing and I used random forest method. It scored about 70% percent on a training set, however when I try to apply my classifier on test set it classifies all points as one class. (in sample size was about 10k points in 3k dimensions with 500,1000 trees)
- Then I figured that I have to do dimensionality reduction, I figured out the amount of features I'm needed (
nFactorslibrary in R) and tried to train random forest classifier again on ICA projections. Prediction on test set again spits out only one class.
- Next thing I figured that maybe my method is not calibrated enough, so I sampled small amount of data and tuned svm classfier on it. Then I trained it on test sample, which resulted in 80% error on training set, but when I predict class on test set I still get one class prediction. Then I also tried create LogitBoost classifier, it classifies 50% of data quite correctly (leaving rest 50% NA's) but when i try it on test sample it results in all NA's.
What am I doing wrong? It looks like my classifier can't catch any underlying structure, though I tuned parameters, increased amount of trees to like 5k and did feature extraction, but still I get only one class prediction. (usually the one that have biggest error) Since other teams are actually doing quite well it means that something gone really wrong.