Duplicated Rows in Mixed Data Type Clustering

I have a dataset which has ~200k rows and looks like the following -

id - ids which are unique col1 - categorical with 3 levels col2 - categorical with 5 levels col3 - numerical col4 - categorical with 5 levels

However, the number of unique rows in this dataset once the id column is removed is only ~1000 rows.

I am trying to use PARTITIONING AROUND MEDOIDS (PAM) clustering in R which requires Gower distance matrix to be calculated. However, calculating this matrix is intensive and the final result will not fit in memory.

I looked at adding weights to each of these duplicated rows when using Gower distance but it looks like from this question - https://stackoverflow.com/questions/21334677/how-do-i-weight-variables-with-gower-distance-in-r, that weights can be set for columns but not rows.

I also looked at Do I need to remove duplicate objects for cluster analysis of objects? but I am not able to find a solution/best practice.

Any suggestions on how duplicate rows are handled in clustering?

• In methods like k medoids or k means duplicated objects clearly affect results. If weighting option is not or cannot be implemented then you are left with the usual problem of too-big-matrix-to-analyze. Try to do your clustering on a random subset(s) of your 200k object dataset. – ttnphns Dec 16 '18 at 9:57

The cluster package is open source, so you can modify it and add support for weighting. I am convinced the authors will be happy to receive such an extension and make it available for future users!