In an 'unguided' experiment, we asked several people to make groups with a defined number of objects presented to them. They were free to do as many classes as they wanted, and to choose their names, but they had to detail precisely the rationale of their groupings.
Their detailed explanations were very useful to us, but we also wonder if there are ways to compare their classifications. There are obvious difficulties: - all classifications have different numbers of classes; - the labels given are also different (the class 'A' of a given observer may correspond to the class '5' of another user).
Even if it seems difficult to consider, is there some mathematical tools or indices to study the 'conformity' between those classifications, and to find which ones are the most similar and the most different? ('Being similar' could mean that most of the time, the objects of two given classifications are in the same classes, even if those classes have not the same name.)
NB: These objects were various kind of silex, and most people classified them according to color, texture/roughness, etc. But in fact, the essential point is that we have for each object (each silex) something like: user 1 classified it in cluster 'A', user 2 classified it in cluster 'red', user 3 classified it in cluster '5', etc. And mathematically, the idea is really to compare a list of cluster attributions between various users.