I have a very large data set that I want to perform classification tasks on. There are about 40 million instances, 16 features, and 2 classes.
I'm attempting to use SciKit-learn
LogisticRegression, but after several hours the processes still have not completed.
I have two questions:
- Is there a way I can estimate the runtime of SciKit-learn classification algorithms? How can I know if the process will complete in minutes, hours, days...?
- Is there a certain algorithm which can scale exceptionally well for large data sets? Is there a library implementing this?