1
$\begingroup$

I am running a hierarchical clustering process in R, using daisyto compute a dissimilarity matrix and agnes for hierarchical clustering, as described in Clustering of mixed type data with R.

With my 8 GB Ram, I constantly run into this error:

Error: cannot allocate vector of size 1.8 Gb

I have 21836 rows with only 2 variables. However, I'd like to use more variables, but I am already running out of memory using only 2.

  • Are there any alternative algorithms for a mixed data set of continuous and categorical variables?

  • Are there any alternative tools (I am currently using R) which would require less memory?

Improve this question
$\endgroup$
7
  • $\begingroup$ You are clustering rows, so the size of the matrix is 21836^2. Multiply by 8 and you'll get roughtly 4 Gb RAM needed only to store the input matrix. But the procedure (and computer) surely needs more free memory to be able to perform. I'm not R user, though, so please wait for somebody knowing R well to advise you. $\endgroup$
    – ttnphns
    May 12, 2015 at 14:18
  • 1
    $\begingroup$ Besides, I'm sceptical about the idea itself - to cluster so many objects by hierarchical cluster analysis (see last point here). $\endgroup$
    – ttnphns
    May 12, 2015 at 14:22
  • $\begingroup$ Thx for sharing your thoughts. Do you have sth in mind how to solve this issue? $\endgroup$
    – rmuc8
    May 12, 2015 at 15:54
  • $\begingroup$ When I have to cluster very many objects by categorical (nominal) variables I use Two-step clustering of SPSS (I'm SPSS user). $\endgroup$
    – ttnphns
    May 12, 2015 at 16:08
  • 1
    $\begingroup$ What algorithm do you apply there? $\endgroup$
    – rmuc8
    May 12, 2015 at 17:36

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.