Problem statement - Idea is to cluster students based on their assessment pattern and score.
Data = I have a data set that contains Binary data of students answers(Correct/incorrect) which i have recoded as 1 & 0. Its a 10*100 matrix of 10 students answering 100 questions. Looks like this
I have tried distance calculation formula of following 2 techniques
Let be the contingency table of binary data such as n11 = a, n10 = b, n01 = c and n00 = d. All these distances are of type d = sqrt(1 - s) with s a similarity coefficient.
- Sokal & Michener formula= (a+d) / (a+b+c+d)
- Jaccard Index formula = a / (a+b+c)
I have found that "Sokal & Michener" works best for me just by looking at the clusters it found and i already knew the pattern in the data very well(i created dummy data) and hence i came to this conclusion of choosing "Sokal & Michener"
But is there a way to statistically verify that 1 algorithm works better then another?
Also can you suggest any other clustering technique for this problem statement?