I often hear that comparing clustering computed on different population is mathematically incorrect. However, I never found any references about it.
Imagine I am working on a dataset such as
household_id sex var1 var2 var3 1 man x y z 1 woman x y z 2 man x y z 2 woman x y z 3 man x y z 3 woman x y z
household_id is a personal identifier of an household, and
var_a set of variables I'm interested in.
I have two related individuals in the household (siblings, couple, parent-children, ...).
I would like to use clustering techniques on the
var_ to come up with a typology.
My issue is this. What if I want to analyse men and women separately. What I would do is to would compute separately for men and women the clustering and then compare the clusters. In cross-tabulating them for example. However, I was told this is wrong. Can you explain me why exactly ?
Moreover, what would be an alternative ? I feel that I am loosing a lot of important differences between the genders in running the clustering on the full database. Also, man and woman are related in this data, and running the analysis on the full database is simply ignoring this household level.
Let me provide an empirical example (with a
Let us imagine I want to compare sequences of men and women
idhouse sex V1 V2 V3 V4 V5 1 1 man a a b c b 2 1 woman a c a a c 3 2 woman a c a a c 4 2 man b c c c a
I will run my sequence analysis and my clustering like :
sq_man = seqdef(df[df$sex == 'man',-c(1,2)]) sq_wom = seqdef(df[df$sex == 'woman',-c(1,2)]) sq_manHAM = seqdist(sq_man, 'HAM') sq_womHAM = seqdist(sq_wom, 'HAM') w1 = hclust(as.dist(sq_manHAM), 'ward') w2 = hclust(as.dist(sq_womHAM), 'ward')
and end up with a typology for men and a typology for women
men = cutree(w1, 2) wom = cutree(w2, 3)
Then for example comparing these with (cross-table)
table(men, wom) wom men 1 2 3 1 2 0 0 2 0 1 2
is not proper. Does anyone knows why ?
library(dplyr) library(TraMineR) df = replicate(5, sample(x = c('a', 'b', 'c'), 10, replace = T)) df = as.data.frame(df) df$idhouse = rep(1:5, 2) df$sex = rep(c('man', 'woman'), 5) df = arrange(df, idhouse) df = select(df, idhouse, sex, everything())