The new data is unlabeled. The training and test data were labeled so I could calculate the model accuracy. The model has made some predictions on the new data. How do I know these predictions are accurate? I want to calculate a margin of error for the prediction accuracy.

Someone up the chain needs to be explained that for a fraction of the cost of a SAS license, you can install R Studio Server on an AWS Virtual Machine with 16 cores and 256 GB RAM - in the hands of 2 good R programmers, that's more powerful than anything SAS can do. Think how fast billion records can be fuzzy matched to each other ! Or for that matter even Open source PostgreSQL with Python + PERL will achieve at a fraction of a cost.

@rolando2 I agree. It is not true. absolute difference does not play much role.

thanks for the tip. will try complete linkage too though given the size of my dataset - wonder if it will take too long to complete.

? But i've tried it with scaling already ! It hasn't worked. My question is what I can do to improve the clustering given that nothing I've tried has worked so far . . .