17
$\begingroup$

I have a multivariate dataset that changes over time. I have extracted (and normalised) some features and used k-means to generate clusters over the entire span of the dataset.

Now I want to see whether the clusters change significantly over time. So, working backwards, and thus reducing the dataset by x-months, can I see a significant reduction on certain clusters?

This, I think, could fall within the realm of time series clustering. I was hoping to avoid complicating the approach, since the clusters are currently meaningful and the approach is relatively simple.

Could anyone please advise me on how to go about this?

My intuition is to reduce the dataset by x-months and then cluster (using k-means) the data for comparison. However, I may be breaking the rules here, and oversimplifying a complicated problem.

$\endgroup$
8
  • 1
    $\begingroup$ Clustering time series is meaningless. $\endgroup$
    – gui11aume
    Aug 10, 2012 at 22:33
  • $\begingroup$ I've read this paper yet it was written in 2005. So I was wondering has there been any advances or is that the final word on things....over and out, if you will :-p $\endgroup$ Aug 10, 2012 at 22:59
  • 2
    $\begingroup$ Consider something more modern than k-means from the 60s. If you want to do time series clustering, read up on time series clustering methods! $\endgroup$ Aug 11, 2012 at 2:46
  • 1
    $\begingroup$ I am reading up on it, and have applied different approaches to the clustering, including PCA (for dimensionality reduction) and aglomerative hierarchical clustering. But I was seeking some advice for the community, it's a broad topic of research and is only a small part of an application I am developing. $\endgroup$ Aug 11, 2012 at 6:24
  • $\begingroup$ The initial problem is how to define a reasonable distance metric for time series, and for that there are many approaches. Discussion here: stats.stackexchange.com/questions/185912/… $\endgroup$ Nov 7, 2016 at 13:25

1 Answer 1

0
$\begingroup$

Time-series clustering requires sample size remaining the same but the features changes over time, otherwise it makes little sense. In the question though, inferring from the description sample size increases over time. In that case, to see significant reduction on certain clusters, one should use a fixed sample-size. Then choose fixed sample from the initial time period, and see how their cluster sizes and memberships are changing over time.

Symbolically, let's say you have 3 datasets (feature matrices) over time:

$$X_{t_{0}} \supset X_{t_{1}} \supset X_{t_{2}}$$

and corresponding clusterings $C_{0}, C_{1}, C_{2}$, where $C$ is essentially instances and cluster membership tables. To judge how clustering changes, take samples at $t_{0}$, such that $X_{0} \supset X_{t_{0}}$. Tracking how $X_{0}$'s membership and cluster sizes on different clusterings $C_{0}, C_{1}, C_{2}$ changes. This would give a good idea if there are "reductions" (significant changes) over different clustering, given that $X_{0}$ is representative over-time.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.