# Appropriate clustering techniques for temporal data?

I have temporal data of activity frequencies. I want to identify clusters in the data that indicate distinct periods of time with similar activity levels. Ideally I want to identify the clusters without specifying the number of clusters a priori.

What are appropriate clustering techniques? If my question does not contain enough information to answer, what are the pieces of information that I need to supply to determine appropriate clustering techniques?

Below is an illustration of the kind of data/clustering I am imagining:

• The plot looks smoothened (interpolated) to me. That is probably misleading. And "longitudinal" I associated with geodata, but apparently you are looking at a time series? – Has QUIT--Anony-Mousse Aug 27 '12 at 6:18
• Don't pay too much attention to the plot, it is just an example. What I want to achieve is the identification of distinct episodes of time based on variables that vary across time. Longitudinal, in my mind, is the same as temporal data, see e.g. en.wikipedia.org/wiki/Longitudinal_study – histelheim Aug 27 '12 at 12:34
• Because in clustering, you will see this term mostly as in en.wikipedia.org/wiki/Longitude - from your question it is not clear what you want to cluster. You can cluster e.g. intervals of time that behave similar across "subjects", or subjects that show the same progress over time. – Has QUIT--Anony-Mousse Aug 27 '12 at 13:32
• I have changed 'longitudinal' to 'temporal' to avoid confusion. Using your words, I think I want to cluster intervals of time. However, it is important to me that the clusters are distinct, continuous episodes in time. – histelheim Aug 27 '12 at 14:26
• Searches with "time series segmentation" or "regime switching models" keywords may help you. – Yves Aug 27 '12 at 16:20

## 5 Answers

From my own research it seems that Gaussian Hidden Markov Models might be a good fit: http://scikit-learn.org/stable/auto_examples/plot_hmm_stock_analysis.html#example-plot-hmm-stock-analysis-py

It definitely seems to find distinct episodes of activity.

• Don't you have to know how many hidden states there are ahead of time? Is there a way to get around that? – JCWong Aug 28 '12 at 4:51
• @JCWong I think you can use a nonparametric Bayesian variant (the infinite hidden Markov model) to avoid that. – jtobin Aug 28 '12 at 20:43
• After long though: HMM doesn't seem to cluster/group events temporally (what it looks from the figure). But, what has been asked is how to get temporal clusters? I am just curious, as I am working on temporal clustering stuff. – RussellB Jun 28 '18 at 13:47

Your problem sound similar to one I'm looking at and this question, which is similar, but less well explained.

Their answer links to a good summary on Change Detection. For possible solutions, a quick google search found found a Change Point Analysis package on Google code. R also has some tools for doing this. The bcp package is pretty powerful and really easy to use. If you want to do it on the fly as data comes in, the paper "On-line changepoint detection and parameter estimation with application to genomic data" describes a really sophisticated approach, though be warned that it's slightly challenging. There's also the strucchange package, but this has worked less well for me.

Wavelets could help you identify periods with different properties. However I'm not sure if there are methods that would divide your timeseries into discrete periods for you. And it seems like there's a lot of theory to wade through, which I'm only at the start of. I look forward to reading other suggestions..

A free introductory book chapter on wavelets.

An R package for significance testing with wavelets.

Have you seen this page: UCR Time Series Classification/Clustering Page?

There you can find both: the datasets to practice on and published results - to compare performance of your own implementation (there is a link on known performance of well known machine learning techniques too). In addition, this page is citing a critical mass of papers from which you could go further on with research for the best approach which suits your problem, data, or needs.

Also, there is another way to do that (potentially) by application of sequitur http:// sequitur.info. If you will be able to normalize/approximate your data well, it will give your grammar of those "distinct periods of time with similar activity levels" see this paper and search for another one, cause I am unable to add more links...

• Could you provide a brief summary of the resources available on this page? – chl Nov 28 '12 at 16:12
• sure i can. starting from there I coded my own classifier – seninp Nov 28 '12 at 18:53

I think you may use Dynamic Time Wrapping to look for similarities between different time series. In order to do that, you may need to discretize your wavelet into collections, like an array. But the granularity would be a problem and if you have a great number of time series, the computation cost will be pretty big to calculate the DTM distance for every pair of them. So you might need some preselection to work as labels.

Check this out. I m also working on some task like yours and this page helped me some.