5
$\begingroup$

I want to fit a Gaussian Process with about 50,000 training examples and 130 features using Scikit-learn. Right now, I only have 1 theta hyperparameters as I run the process with all defaults. But I get a MemoryError when it tries to compute the covariance matrix.

This is where I get the error https://github.com/scikit-learn/scikit-learn/blob/master/sklearn/gaussian_process/gaussian_process.py#L52

What can I do?

|cite|improve this question
$\endgroup$
  • $\begingroup$ How much RAM do you have? Perhaps you need more. Are all your features numeric? I'm not familiar with scikit-learn, but I know that in R if you accidentally include a factor variable with many levels in your model it can cause you to run out of RAM. $\endgroup$ – Zach May 22 '13 at 13:48
  • $\begingroup$ I have 16GB of RAM. They're all numeric. I can run it with 2% of my training data (~1000 examples), which is less than ideal. $\endgroup$ – siamii May 22 '13 at 13:50
  • 1
    $\begingroup$ It looks like python is trying to create 50,000 x 25,000 = 1.25e9 row by 2 column array. This works out to an array with 2.5e9 elements. Assuming integers in python take 32 bits of RAM, thats roughly 9.3 GB of RAM. It possible that you've already used 7GB of RAM by the time you get to this point in the code. I don't think there's much you can do, except reduce the training set size, get more RAM, or try a different model. $\endgroup$ – Zach May 22 '13 at 14:06
  • 1
    $\begingroup$ Furthermore, the next line (53) is going to cause you even more problems, when it tries to allocate a 1.25e9 x 150 element array. By my calculations (which could be incorrect) this array will require almost 700GB of RAM! $\endgroup$ – Zach May 22 '13 at 14:10