I am working with a data-set of around ~100000 observations(rows) and ~256 features(columns). Is there any recommendation for applying Machine Learning techniques on such a data-set efficiently ? Maybe by parallelization or similar approaches ?

I am currently using Matlab for applying different Machine Learning, but have investigated Python's scikit-learn as well for applying:


  • Gaussian processes for regression GPR


  • Linear discriminant analysis LDA
  • Support vector machine SVM

Obviously dimentionality reduction comes to mind, however for this specific data-set removing some of the features or applying transformations will distort the information.

  • 1
    $\begingroup$ That's less than 200 MB of data in MATLAB double precision. That doesn't sound very big to me. $\endgroup$ – Mark L. Stone Dec 10 '17 at 18:19
  • $\begingroup$ @MarkL.Stone The PC is crashing every time i run GPR on the full dataset, or it just runs for hours without any results at the end and i would have to stop it manualy $\endgroup$ – AnarKi Dec 10 '17 at 18:21
  • $\begingroup$ Lots of options and methods (algorithms) available mathworks.com/help/stats/fitrgp.html . Some might make a big difference in run time and success prospects. Have you set verbose to 2 so that you can see intermediate output, progress (or not) being made? $\endgroup$ – Mark L. Stone Dec 10 '17 at 18:30
  • 1
    $\begingroup$ Your PC is crashing because out of the box gaussian process regression is $\mathcal{O}(n^3)$ (it requires the inversion of the gram matrix). If you want to scale gaussian processes to more than a few thousand data points you need to look into approximate inference methods that specifically can handle 100k data points. $\endgroup$ – aleshing Dec 10 '17 at 18:52
  • 2
    $\begingroup$ You problem doesn’t qualify as a big data problem. I suggest revising the title of your question. $\endgroup$ – aivanov Dec 10 '17 at 19:00

Try Gradient Boosting Machine (GBM) and you’ll get results within minutes.

example with scikit-learn

Also check an extremely fast GBM implementation with R and Python bindings: xgboost


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.