I'm new to machine learning models and do not have an opportunity to run real business models.

Doing some studies on classification models using SK-learn (SVM, Logistic, tree based models and etc.) I wonder if I have really large data size to do the training, would it still be a matter what model I choose to fit the data and then use on test data? My naive guess is the more data you have, the less likely the type of the model matters. Also, if we really have huge amount of data, we probably cannot run many alternative models to see which one is better, because it may take days to fit even just one model.

Therefore my naive conclusion is, in real business where people running model against their huge customer base, they won't have the needs of trying many model options or grid search many parameter settings but still can come up well performed model.

Is my naive understanding valid? :)


Model selection definitely matters. If you choose a model with high bias, all the data in the world will not improve the fit. For instance, no matter what the training size is, a linear regression won't be able to fit data generated from a quadratic process (and yes I know you can include a quadratic term, but what happens when you have hundreds of variables? How will you know what kinds of terms to include?).

| cite | improve this answer | |
  • $\begingroup$ Thank you Demetri for your answer. I wonder what people do in real business, with super large data. Do you think they pre-select the model with smaller sampled data set to play with fancy models and tons of tuning before applying the best choice onto the real large one? $\endgroup$ – MeiNan Zhu Nov 13 '18 at 4:07
  • $\begingroup$ In my experience, highly flexible models are used with sampled data. There is not a ton of time to play with parameter tunings. Often times, good enough is good enough. Keep in mind that companies often have HUGE compute to throw at a problem. They aren't making models on their laptop. $\endgroup$ – Demetri Pananos Nov 13 '18 at 4:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.