4
$\begingroup$

I have a large training set, and it is too big to apply some algorithms due to computation limits.

What are the common methods to decrease training set size without losing significant amount of information?

Edit:

Training examples have 3 features and it is a 0/1 classification task.

Improve this question
$\endgroup$
10
  • 2
    $\begingroup$ How large is the training set? What algorithms do you want to apply? What software are you using? Are the three features categorical or continuous; if categorical, how many values can they each take on? Finally, are your 0/1 cases relatively balanced or is one class relatively rare? All these details may help people give you more useful advice. $\endgroup$
    – Anne Z.
    Commented Feb 5, 2012 at 14:13
  • $\begingroup$ @metdos, could you please post a few lines of your data, and/or the results of training with say the first 100k, second 100k ... samples ? Also 3 is nice to visualize, google "visualize 3d point clouds". $\endgroup$
    – denis
    Commented Feb 7, 2012 at 18:26
  • $\begingroup$ Features are continuous, I have approximately 400K examples in training set. I tried SVM with Matlab. $\endgroup$
    – metdos
    Commented Feb 7, 2012 at 20:25
  • $\begingroup$ @metdos, linear kernel ? How good was the classification, how long did it run -- 400k x 3 doesn't seem large. $\endgroup$
    – denis
    Commented Feb 8, 2012 at 15:11
  • $\begingroup$ @Denis yes it is linear kernel, matlab gives me "Out of memory" error and on net it says it needs n^2 continues memory. Do you suggest any other tool or library(C++, Java, Python) ? $\endgroup$
    – metdos
    Commented Feb 8, 2012 at 18:47

2 Answers 2

Reset to default
5
$\begingroup$

The brief answer is random sampling, but the more difficult issue is determining the size of the random sample that you should use. One efficient solution to that problem is provided by progressive sampling—a method that Foster Provost, Tim Oates, and I developed in the late 1990s [1]. The approach begins with a small sample size and increases sample size according to a sampling schedule, checking whether model accuracy increases at each iteration. We show that a geometric schedule (e.g., doubling the sample size on each iteration) is asymptotically no worse than knowing the correct sample size in advance.

[1] F. Provost, D. Jensen, and T. Oates (1999). Efficient progressive sampling. Proceedings of the Fifth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. http://pages.stern.nyu.edu/~fprovost/Papers/progressive.ps

Improve this answer
$\endgroup$
0
1
$\begingroup$

I believe you need to be more specific. What are you trying to do/classify? How many classes do you have? Usually the training set is too valuable to dismiss. Have you thought of reducing the dimensionality, this is usually a must-do when you have many attributes and it will make computations much faster.

One thing I know people have done is to select x number of random data points from each class, but again it depends on the problem. You want to make sure you don't add bias to your new training set.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.