I'm performing analysis for a colleague who has data from two biological conditions. Since clustering is a standard in our field, she'd like to perform clustering on each condition separately, with the goal of identifying datapoints that change clusters between conditions. For example:
Here, we'd want to flag datapoint 3 as changing clusters. This seems straightforward (look how simple the example is!).
However, there's extra complexity because I'm required to choose the number of clusters a priori, and this changes the result. For example if there's one cluster, datapoint 3 would not be flagged in the example above; if there are three clusters, other datapoints would also be flagged. So, is clustering a viable/appropriate method for doing what we want? If so, is there a conventional method that doesn't require choosing the number of clusters beforehand?