2
$\begingroup$

I have a data set consisting of roughly 170,000 input vectors having 3,000 features each. On this data set I would like to perform a Kernel PCA using scikit-learn. Unfortunately, any attempt always results in:

MemoryError

I'm on a PC having 32 GB of RAM and setting the copy_X parameter to 'false' doesn't help. Any suggestions?

$\endgroup$

closed as off-topic by kjetil b halvorsen, Jan Kukacka, Peter Flom May 28 '18 at 13:12

This question appears to be off-topic. The users who voted to close gave this specific reason:

  • "This question appears to be off-topic because EITHER it is not about statistics, machine learning, data analysis, data mining, or data visualization, OR it focuses on programming, debugging, or performing routine operations within a statistical computing platform. If the latter, you could try the support links we maintain." – kjetil b halvorsen, Jan Kukacka, Peter Flom
If this question can be reworded to fit the rules in the help center, please edit the question.

2
$\begingroup$

Though SO seems to be a better fit for this question, there is a theoretical knowledge of the algorithm to keep in mind.

On Wikipedia there is a specific paragraph related to large samples. Just like (kernel) SVM, you need to compute the whole matrix $K(x_i, x_j)$. Where $x_i$'s are your sample points. You have 170 000 of them, so 170 000 ^ 2 terms to compute (and store) in the matrix $K$. Even with enough memory, I doubt the calculation would end.

An approach could be (from wikipedia) :

One way to deal with this is to perform clustering on the dataset, and populate the kernel with the means of those clusters. Since even this method may yield a relatively large K, it is common to compute only the top P eigenvalues and eigenvectors of K.

Or to look for streaming implementations of KPCA.

$\endgroup$
  • $\begingroup$ Thank you very much for your helpful reply. However, I'm still looking for a way to direclty apply the theory. Are there any functions I could use in scikit-learn, Python in general (or any other framework) to automatically cluster the data first? $\endgroup$ – Hagbard May 28 '18 at 8:33
  • $\begingroup$ @Hagbard as far as I know, there is no such thing... $\endgroup$ – RUser4512 May 28 '18 at 9:36

Not the answer you're looking for? Browse other questions tagged or ask your own question.