Thanks in advance for the help
I have around 13000 samples with two features each and I would like to cluster these samples into groups. A few caveats. One, I don't know how many groups there are (there could be as little as 70 to as many as 1000), but I do know a few things about the groups. The groups should be very, very similar across the two features. It is possible that several groups might be "close" to each other though. Two, there won't be any uniformity in terms of the number of elements in the groups. It is very possible (and from my knowledge of the dataset very likely) that some groups will have on the order of a hundred(s) of elements while others might have only a handful (possibly even one). It is not as important to identify the groups with a small number of elements, but erroneously adding them with a larger group could create noise for what I plan to do with these groups (this is not pertinent to this problem, I include this only as a constraint). Therefore I would like a method that will tend to err on the side of caution so to speak. Put simply, I would like to create a new group for any sample, or small set of samples, when it is not immediately clear that they belong to another group (when in doubt, just create a new group). I expect there to be many more groups with a low number of elements then groups with large elements. Hopefully this illuminates what I would like to accomplish accurately.
I believe the proper method to assigning these samples to different groups would be some sort of unsupervised clustering approach. However, I have very little experience with clustering. My questions are the following.
Is clustering the right approach here? If not, how else might I go about this problem?
Assuming that clustering is the right approach here, what method should I use? I understand that clustering algorithms often require the number of clusters in which to divide the samples. However, I don't even know the order of magnitude of the number of samples. Obviously, I could just create a 2d plot of the my samples and give a guesstimate, but I'm hoping for a method that would be a tad more rigorous and methodical then giving a guess. Are there any methods then that can dynamically create different numbers of groups given an input dataset and a set of tolerance settings?