We are analyzing temporal behavioral patterns across many users and we want to cluster users in order to understand "natural types of behavior". Our idea is to represent the data (672 bins for each user) using a discrete-time Fourier transform. Then the behavior of each user will be decomposed into a combination of basic behavioral signal types.

We will then apply Gaussian mixture modelling (GMM) for clustering the users based on their coefficients for the different signals in the Fourier representation. Now, some of these basic signals will be very small, while others will be quite big. Of course, I am more interested in finding larger differences in the behavior rather than small differences. However, if I just use the coefficients directly, the GMM will have no way of differentiating between the big signals and the small signals.

How do I get this information from the Fourier transform and input into the GMM in order to cluster primarily based on large differences in behavioral signals? I'm brainstorming that I might scale the coefficients by the amplitude or something similar. However, perhaps the theory behind the Fourier transform can propose a better way?


Interesting. I've been working on a similar question recently. First, as is well known, the full periodogram isn't needed since the Nyquist frequency is N/2. Then, if you rank the set by decreasing Amplitude, you could select that subset of frequencies that best represents the periodicities. I'm not aware of any theoretical motivations for selecting a cutoff...but a judgement-based heuristic would work.

My approach was a little different, expanding on the FFT. Obviously, there is likely to be some autocorrelation to the time series as well. Why not fold in a measure to capture that, e.g., the DW or related metrics? Then, there can be issues with trend, cointegration and unit roots...why not use measures such as the Augmented Dickey-Fuller or Philips-Perron to express those relationships? Then, Rob Hyndman put out a paper on clustering time series a couple of years ago that decomposed the time series into the moments of the distribution, and clustering on those metrics.

Anyway, those are some thoughts...

  • $\begingroup$ The paper mentioned I believe is Characteristic-Based Clustering for Time Series Data. IIRC, the Ramsey book on Applied Functional Data analysis had some examples of using PCA to do clustering as well. $\endgroup$ – Andy W Oct 9 '15 at 11:59
  • $\begingroup$ @AndyW Thanks! That is the reference. Your mention of PCA reminded me of another issue...dealing with extreme values or heavy-tailed data. Additional metrics could include estimating the tail index for each time series. To the extent that the distributions are heavy-tailed, this metric would help identify that. In addition, Nissam Taleb's book Silent Risk (freely downloadable from his website) has a number of other metrics that go beyond the moments of the distribution to further aid in decomposing the extremas, if they exist. $\endgroup$ – Mike Hunter Oct 9 '15 at 12:09
  • $\begingroup$ Finally, PCA is linear. There are robust methods now such as Cauchy PCA that do not make gaussian assumptions. Check out Pengtao Xie's paper link[link] $\endgroup$ – Mike Hunter Oct 9 '15 at 12:12
  • $\begingroup$ Thanks for the thoughts. Is there a metric that shows how much of the variation in the data I am capturing by representing the data using e.g. the top 50 basic signals ranked by amplitude? $\endgroup$ – pir Oct 9 '15 at 12:14
  • $\begingroup$ PCA would give you the eigenvectors and the proportion of variance explained based on them... $\endgroup$ – Mike Hunter Oct 9 '15 at 12:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.