1
$\begingroup$

I'm building a model based on a database with around 90000 observations and 100+ variables. My target variable is binary (0 or 1).

I'm using SAS Miner and I want to test a few high performance techniques.

I already tested Random Forest and got very successful results.

Now, I'm giving a try at using Support Vector Machine (SVM) and Radial Basis Function (RBF). However, I'm experiencing a lot of issues running those procedures those due to insufficient memory.

I researched a bit about both procedures but I still have no clue if It's possible to run SVM and RBF with such large data.

If that's not possible, when should I use SVM and RBF?

Thanks in advance!

Edit 1: I was able to run SVM with 22000 observations database and a small amount of variables (five). Still, no success in performing with the bigger data and RBF.

$\endgroup$
1
$\begingroup$

I am not familiar with SaS, but Kernels consume a good amount of memory, you may try dimensionality reduction with PCA, in order to improve memory utilization. Stochastic gradient descent shines in bigger datasets, I mean a neural network. Another recent option worth exploring is Gaussian Process which also uses kernels, I have better results with GP than Random Forest, there is a combination of variational inference for GPs that aims for cases like yours, one library to check is Gpflow, which relies on Tensorflow, well this is Python. I believe if you want to scale and have recent algorithms Python is a must, I hope this helps.

$\endgroup$
  • $\begingroup$ Thanks for the info! I'll give that a try and report my findings. $\endgroup$ – Rovere Apr 11 '18 at 20:43
  • $\begingroup$ What If I split my data into multiple different sets, run SVM for each one of them and then run an ensemble? Would that be a good idea? $\endgroup$ – Rovere Apr 12 '18 at 16:57
  • $\begingroup$ Take a look to this:blog.kaggle.com/2016/12/27/… $\endgroup$ – Germán Alfaro Apr 12 '18 at 17:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.