5
$\begingroup$

I have relatively large (100k items) dataset which I need to split in two groups. So far I've tried knn and the results are not good mainly because I have disproportion in my training data: 90% of points belong to the first group. The same proportion is expected to be in test data.

Is there a way to improve prediction quality with this kind of data? Performance is not important while quality of prediction is paramount.

$\endgroup$
8
  • $\begingroup$ do you mean 'into' groups or 'in two' groups?. Also how many variables do you have? $\endgroup$
    – user603
    Commented Feb 21, 2013 at 13:25
  • $\begingroup$ @user603 I mean in two groups say spam and not spam. I have <100 features, but it may grow up to 1000 features (other people working on implementing new features). $\endgroup$
    – Moonwalker
    Commented Feb 21, 2013 at 13:29
  • $\begingroup$ This thread is somewhat related, & may be interesting for you: does-an-unbalanced-sample-matter-when-doing-logistic-regression. $\endgroup$ Commented Feb 21, 2013 at 14:15
  • $\begingroup$ wait: this is confusing. Do you have access to the real labels? --i wrote my answer assuming that you have no access to the true labels. $\endgroup$
    – user603
    Commented Feb 21, 2013 at 14:17
  • $\begingroup$ @user603 for a few datasets yes, I have access to real labels (I'm using separate datasets for training and testing), but not for the ones used in production. Sorry for confusing you, I really liked your answer $\endgroup$
    – Moonwalker
    Commented Feb 21, 2013 at 15:13

1 Answer 1

1
$\begingroup$

First of all, if you ditch accuracy for AUC and use a k-NN implementation that outputs some continuous score (proportion of votes, weighted votes, etc) you would be able to know if your model has any discriminant power.

Now, if you want to keep accuracy, you could try different weights to the votes of each class.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.