# Sequence analysis - Clusters quality - Time Use

I am trying to run sequence clustering on time use data but I fail to have "acceptable" clustering solution according to Studer (2010).

The sequences have 76 episodes of 15 minutes slots (12 states). The data recorded people's activities during one day. The sequences are quite complex (lots of different number of states, different timing, ...).

I was wondering at what point the clustering solution criterions can take into account the "natural messiness" of data. In comparison with Life Course data, Time Use data are much more fuzzy and unpredictable. Because, even though, the clustering solution criterions are quite low, the visual exploration is comprehensive and patterns are easy to detect and understand. How should I interpret the fact that I can easily understand "what is going on" in the data even though the criterions are telling me that there is no pattern ?

I am using the great library('WeightedCluster') for computing clusters quality :

> as.clustrange(wardCluster, diss=mcdist.cost, ncluster=15)
PBC   HG HGSD  ASW ASWw    CH   R2  CHsq R2sq   HC
cluster2  0.29 0.35 0.35 0.12 0.13 15.07 0.04 29.27 0.08 0.32
cluster3  0.27 0.34 0.34 0.06 0.07 13.46 0.07 23.84 0.12 0.31
cluster4  0.16 0.21 0.21 0.00 0.01 12.11 0.09 20.43 0.15 0.36
cluster5  0.24 0.33 0.32 0.01 0.02 11.49 0.11 21.09 0.19 0.29
cluster6  0.28 0.39 0.39 0.02 0.04 10.68 0.13 20.46 0.22 0.26
cluster7  0.30 0.44 0.44 0.03 0.05 10.14 0.15 19.67 0.25 0.24
cluster8  0.30 0.45 0.45 0.03 0.06  9.72 0.16 18.76 0.27 0.23
cluster9  0.33 0.54 0.54 0.04 0.07  9.42 0.18 18.56 0.30 0.20
cluster10 0.35 0.59 0.59 0.05 0.08  9.09 0.19 18.03 0.32 0.18
cluster11 0.28 0.51 0.51 0.01 0.04  8.84 0.20 17.21 0.33 0.21
cluster12 0.29 0.54 0.54 0.02 0.05  8.55 0.21 17.09 0.35 0.20
cluster13 0.30 0.59 0.59 0.03 0.06  8.31 0.22 16.65 0.36 0.18
cluster14 0.31 0.61 0.61 0.03 0.07  8.06 0.23 16.22 0.38 0.18
cluster15 0.31 0.61 0.61 0.04 0.07  7.80 0.24 15.99 0.39 0.17


The silhouette is very low as well as the R2. Would you recommend a particular measure that is well fitted for "messy" data ?

How should I argue in a scientific paper that I can (as a researcher) identify clear patterns but that the quality measures can not ?

Another issue (related to the preceding) is that I would like to "control" more finely the clustering solution. Let me explain.

There is some variables that are structuring the clustering solution, but not always in the way I want to. In this case (as it is often the case also in Life Course), the structuring variable is the presence of children in the household.

What happen is this : all the people with children are gathered in one single cluster, and the rest is spread among the other clusters. Even if I increase the number of cluster, I can not "break down" this one single "parent" cluster. I know that in the seqtree (regression tree) it is possible to input variables in order to "control" the split of the tree. Is it possible to do so with traditional clustering methods ?

Because I might think that if it is possible to define the known "great" groups that structure the data, it would help improve the homogeneity of the clusters.

(I am not sure that can I split the data, for example by Gender or by Children in Household, ran different clustering and then put back the different solutions together and ran logistic regression. I have been told that I could not do it because apparently we can not compare anymore the clusters).

The goal in the end is to use the clusters in a (logistic) regression model.

So because the sequence clustering needs to be done on the whole sample, I was wondering if I could do something like different "round" of clustering. One first round on the whole sample and then inside each cluster.

For example if I want the clustering solution to cluster around the presence of children, would it be possible to run first a solution on the whole sample and then re-run a clustering solution inside the "parent" clusters for whom I want more clusters ?

Would this be acceptable ? Could I still then compare the quality of "second round" clustering with the first ? Do you think that would help improve the R2 or the silhouette ?

Would I be able to use these two clustering solutions in the same regression ?

Lesnard and Kan (2011) did something bit similar but did not use regression ("two-stage OM"). Do you have some scientific litterature to recommand ?

Ref

Lesnard, Laurent, and Man Yee Kan. "Investigating scheduling of work: a two‐stage optimal matching analysis of workdays and workweeks." Journal of the Royal Statistical Society: Series A (Statistics in Society) 174.2 (2011): 349-368.

Studer, Matthias. Etude des inégalités de genre en début de carrière académique à l'aide de méthodes innovatrices d'analyse de données séquentielles. Thèse de doctorat : Univ. Genève, 2012, no SES 777

• Thank you for your answer. Do you have a reference for the "different criterion" (run a cluster analysis for parent and non parent and then compare the clusters, because the types are build on different criterion.) ? Thanks again. – giac Aug 17 '15 at 10:10