0
$\begingroup$

I am clustering a rectangular data set with 82 records and 2 columns (a date and an integer between 1 and 2). The goal is to cluster dates that are close in proximity automatically in a way that perhaps an analyst could manually achieve by "circling" close data points on a timeline.

In Python/Pandas/SKLearn I am fitting the model to the data as such:

mod = KMeans(n_clusters=15)
mod.fit(df)

This results in a model that I feel fits very well to the data and "makes sense" to the eye when plotted. I am attempting to recreate my results in SPSS using the KMeans clustering tool with the following parameters using both variables (date/integer) just like the model I fit in SciKit-Learn:

Number of Clusters = 15
Method = "Iterate and Classify"
Maxiumum Iterations = 300 (copied from defaults from SKLearn KMeans)
Convergence Criterion = .0001 (copied from defaults from SKLearn KMeans)

SPSS fits the data in a far less (at least it appears to the eye) optimally. In Python the dates are converted to numeric using Pandas.to_numeric and then converted back after the clusters are created. In SPSS I have tried clustering with and without converting the dates to numeric formats.

Am I doing something wrong? I would certainly (perhaps, incorrectly) that SKLearn is trustworthy using out of the box settings and only modifying the n_clusters parameter. Do I have something wrong in SPSS?

--Also: if this exchange is not appropriate for my post and better suited to StackOverflow, please let me know!

$\endgroup$
  • $\begingroup$ This is going to be quite tricky to answer as it demands a knowledge of both SPSS and scikit-learn. If you do not get answers here you might try forums which specialise in one or other of those softwares. $\endgroup$ – mdewey Sep 18 '18 at 15:52
  • $\begingroup$ Please show your data. How can dates (which are both progressive and circular) be processed by k-means, I just wonder. $\endgroup$ – ttnphns Dec 17 '18 at 1:01
  • $\begingroup$ @ttnphnsI am converting the dates to numeric data types as such df['Date'] = pd.to_numeric(df['Date']) $\endgroup$ – Patrick Flynn Dec 17 '18 at 17:01
0
$\begingroup$

I do not know SPSS at all, but KMeans actually has a couple of common variations. scikit-learn KMeans defaults to the KMeans++ initialization method,which has been shown to be more robust in practice. The original Kmeans uses random cluster initialization, so if SPSS uses that, it could explain the differences. There might be an option to switch cluster initialization method in SPSS.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.