1
$\begingroup$

I am running a hierarchical clustering process in R, using daisyto compute a dissimilarity matrix and agnes for hierarchical clustering, as described in Clustering of mixed type data with R.

With my 8 GB Ram, I constantly run into this error:

Error: cannot allocate vector of size 1.8 Gb

I have 21836 rows with only 2 variables. However, I'd like to use more variables, but I am already running out of memory using only 2.

  • Are there any alternative algorithms for a mixed data set of continuous and categorical variables?

  • Are there any alternative tools (I am currently using R) which would require less memory?

|cite|improve this question
$\endgroup$
  • $\begingroup$ You are clustering rows, so the size of the matrix is 21836^2. Multiply by 8 and you'll get roughtly 4 Gb RAM needed only to store the input matrix. But the procedure (and computer) surely needs more free memory to be able to perform. I'm not R user, though, so please wait for somebody knowing R well to advise you. $\endgroup$ – ttnphns May 12 '15 at 14:18
  • 1
    $\begingroup$ Besides, I'm sceptical about the idea itself - to cluster so many objects by hierarchical cluster analysis (see last point here). $\endgroup$ – ttnphns May 12 '15 at 14:22
  • $\begingroup$ Thx for sharing your thoughts. Do you have sth in mind how to solve this issue? $\endgroup$ – rmuc8 May 12 '15 at 15:54
  • $\begingroup$ When I have to cluster very many objects by categorical (nominal) variables I use Two-step clustering of SPSS (I'm SPSS user). $\endgroup$ – ttnphns May 12 '15 at 16:08
  • 1
    $\begingroup$ What algorithm do you apply there? $\endgroup$ – rmuc8 May 12 '15 at 17:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.