18
$\begingroup$

I have a question about cluster analysis. There are 3000 companies, which have to be clustered according to their power usage over 5 years. Each company has values for every hour during 5 years. I would like to find out if some companies have the same pattern in usage power over the time period. The results should be used for daily prediction of power usage. If you have some ideas how to cluster time series in SPSS, please share with me.

$\endgroup$
10
$\begingroup$

A) Spend a lot of time on preprocessing the data. Preprocessing is 90% of your job.

B) Choose an appropriate similarity measure for the time series. For example, threshold crossing distance may be a good choice here. You probably won't desire dynamic time warping distance, unless you have different time zones. Threshold crossing may be more appropriate to detect temporal patterns, while not paying attention the the actual magnitude (which will likely be very different from company to company).

C) Cluster the resulting dissimlarity matrix using methods such as hierarchical clustering or DBSCAN that can work with arbitrary distance functions.

$\endgroup$
  • $\begingroup$ can you explain why dynamic time warping distance is no a good option for time series clustering? $\endgroup$ – Hardik gupta Nov 25 '18 at 6:22
  • $\begingroup$ That was not a general statement. Whether it is good or not depends on whether you want to allow time warping, or not. $\endgroup$ – Anony-Mousse Nov 25 '18 at 10:25
6
$\begingroup$

You might want to look at Forecasting hourly time series with daily, weekly & annual periodicity for a discussion of hourly data involving daily data and holidays/regressors. You have 5 years of data while the other discussion involved 883 daily values. What I would suggest is that you could build an hourly forecast incorporating regressors such as day-of-the-week; week-of-the-year and holidays using daily totals as an additional predictor. In this way you would have 24 model for each of the 3,000 companies. Now what you want to do is by hour, estimate the 3,000 models using a common ARIMAX structure accounting for the pattern of response around each of the regressors, the day-of-the-week, changes in the day-of-the-week parameters and weekly indicators while isolating outliers. Then you could estimate the parameters globally using all 3000 companies. Perform a Chow Test http://en.wikipedia.org/wiki/Chow_test for constancy of parameters and upon rejection cluster the companies into homogenous groups . I have referred to this as single dimension cluster analysis. Since SPSS has very limited capabilities in time series you might want to look elsehere for software.

$\endgroup$
  • 1
    $\begingroup$ "vanilla" seems an odd word to use about R; not clear on the translation in more usual R terminology. Any distinction between base R and extra contributed packages from CRAN really doesn't bite experienced users, or even novice ones, as they are equally free and equally accessible. My impression is that someone who had access to SPSS can say fairly easily that something isn't possible in SPSS at present without programming; to say the same about R requires familiarity with all the time series packages. $\endgroup$ – Nick Cox Aug 10 '13 at 19:07

protected by kjetil b halvorsen Aug 26 '17 at 19:51

Thank you for your interest in this question. Because it has attracted low-quality or spam answers that had to be removed, posting an answer now requires 10 reputation on this site (the association bonus does not count).

Would you like to answer one of these unanswered questions instead?

Not the answer you're looking for? Browse other questions tagged or ask your own question.