# Statistical test to show the importance of one cluster

I have one dataset of two variables (x,y). When the data is plotted in a 2D diagram, I see some data points create a good cluster, while the other data points are scattered randomly.

Here is an example:

(These data points were collected through an experiment. The weight is equal among all points. After plotting the data points, I found that some points can create a cluster. Then, I changed the color of these points to green color to show those better.)

Based on the plot, I have two clusters: (1) Green points (2) Red Points. Each data point in both clusters has two variables: X, Y.

My question is that:

How can I conduct a statistical test to (statistically) show that the Green cluster has lower entropy than the red cluster?

• H0: No differences

• Ha: Green cluster is statistically better (Less Entropy) than the red cluster.

I actually want to show that among all data points (Red + Green), the green data points create a good cluster.

• How were the clusters identified, exactly? – whuber Aug 4 '16 at 22:20
• These data points were collected through an experiment. The weight is equal among all points. After plotting the data points, I found that some points can create a cluster. Then, I changed the color of the points with green color to shows those better. I want to discuss that some data points (green data points) can create a cluster. To this end, I need to statistically show what I told in the question. – Harry UNL Aug 5 '16 at 3:19
• Because the identification of the cluster was subjective (albeit, no doubt, well informed by expertise) and irreproducible, no statistical test can provide reliable results--and in particular, you cannot sanctify your judgment by applying any such test. Since it is self-evident that the green cluster has much lower entropy than the red points, why not just compute and report the resulting entropies? – whuber Aug 5 '16 at 12:15

It's more about the ability to apply an algorithm which can discern the two clusters. Yes, you can see the two clusters, but typical statistical tests of e.g. means won't help since objects colored green are surrounded by red. Thus, try K-nearest neighbors (kNN), with $k=3$. This would essentially involve selecting each object one at a time, finding out the majority class (e.g. color) among the 3 closest neighbors of the object based on Euclidean distance, and then assigning the majority class to the object. (for example, if 2/3 of the 3 closest neighbors are green, then you assign the predicted class of green to the test object). Here, you are predicting class membership based on the majority class (green or red) among the 3 closest neighbors. Simply increment by 1 the elements in a confusion matrix to determine classification accuracy, call this $A$. Since you know the true class (colors shown), you can rerun the same steps but only this time randomly shuffling the true class feature (red/green) over all objects. Repeat this $B=10000$ times, calculate $A^*$ for each iteration, and the number of times the classification accuracy of objects after shuffling (the color feature) $A^*$ exceeds $A$ divided by $B$ is the statistical p-value. This is merely called a randomization test, or empirical p-value testing.