Best Scalable Classification Algorithms

Question

I have a very large data set that I want to perform classification tasks on. There are about 40 million instances, 16 features, and 2 classes.

I'm attempting to use SciKit-learn LinearSVC and LogisticRegression, but after several hours the processes still have not completed.

I have two questions:

Is there a way I can estimate the runtime of SciKit-learn classification algorithms? How can I know if the process will complete in minutes, hours, days...?
Is there a certain algorithm which can scale exceptionally well for large data sets? Is there a library implementing this?

Your first question would be off-topic in here (it is about programming in python), while the second one is on-topic. As about guessing the runtime: run it on smaller data samples and interpolate from it to the bigger sample (notice that this does not to have be linear), see e.g. cs.stackexchange.com/questions/192/… — Tim, Commented Mar 17, 2016 at 8:23
I'd argue that the time-complexity of various algorithms could be on-topic here too. — Matt Krause, Commented Mar 18, 2016 at 21:53

Jacques Wainer · Accepted Answer · 2016-03-18 22:16:08Z

2

Some suggestions

LASVM ==> http://leon.bottou.org/projects/lasvm - an online SVM. Never used

PMSVM ==> https://sites.google.com/site/wujx2001/home/power-mean-svm - a linear SVM. A friend used on 10M data, 200K features.

Logistic regression with SGD http://deeplearning.net/tutorial/logreg.html#logreg the first tutorial in the deeplearning.net on how to use Theano.

answered Mar 18, 2016 at 22:16

Jacques Wainer

5,4522 gold badges23 silver badges34 bronze badges

Add a comment |

Diego · Accepted Answer · 2016-03-19 11:09:08Z

1

SGD Classifier should fit your purpose. With your amount of data and n_iter=5 by default it could converge in an hour or so.

But try on a subsample first to gauge. For other classifiers too.

answered Mar 18, 2016 at 21:23

Diego

4443 silver badges9 bronze badges

Add a comment |

2 Answers 2