1
$\begingroup$

I am working with cyclical data (Days 1-7, hours 1-24). I want to project it into a feature space that can understand that 1 and 7 are close days and 1 and 24 are closer than 22 and 24, etc, and then I will do k-means clustering using kkmeans in R.

Does a Guassian RBF accomplish this? I suspect not. If not, can someone help me think of a kernel, preferably one from this list (w/ modified parameters possibly) so I do not have to reprogram it?

rbfdot Radial Basis kernel "Gaussian"
polydot Polynomial kernel
vanilladot Linear kernel
tanhdot Hyperbolic tangent kernel
laplacedot Laplacian kernel
besseldot Bessel kernel
anovadot ANOVA RBF kernel
splinedot Spline kernel
stringdot String kernel

It seems like it might be something like (for days)

d(x,y) = min(d(x,y),d(x+7,y))

Is this possible to program into k-means package you think? It seems difficult to modify as a lot is written in FORTRAN

$\endgroup$
2
$\begingroup$

What about using a cyclical function such as say a sine wave as kernel? So your kernel transformation would be something like:

$ f = 24 \\ t = (x_1 - x_2)\\ k(x_1,x_2) = \sin(t * \pi / f) $

Of course there is a sligth assymetry here between the distances, even better would be to use triangle wave function:

$ f=24 \\ t = (x_1 - x_2)\\ k(x_1,x_2) = | 2 t/f - \lfloor t/f + 0.5 \rfloor | $

Which reveals the following graph when plotting similarity for say hour 0:

enter image description here

Code to generate this plot in python:

import matplotlib as plt
import numpy as np

def kernel1(x1,x2):
    freq = 24.0 #hours
    t = x1-x2
    value = np.abs(np.sin(t*np.pi/freq))

    return value
def kernel2(x1,x2):
    a = 24.0
    t = (x1-x2)
    value = np.abs( 2.0* (t/a - np.floor(t/a + 0.5)) )    
    return value 

hours = np.array(range(0,47))
hourtocompareto = 0
similarities1 = [kernel1(hourtocompareto,hour) for hour in hours]
similarities2 = [kernel2(hourtocompareto,hour) for hour in hours]

plt.plot(hours,similarities1,color="blue")
plt.plot(hours,similarities2,color="red")
plt.ylabel('Similarity')
plt.xlabel('Lag')
plt.show()
$\endgroup$
  • $\begingroup$ This seems to be it, thank you!... if I can manage to figure out how to program it into kkmeans :( $\endgroup$ – wolfsatthedoor Nov 27 '14 at 6:23
  • $\begingroup$ Is it possible to transform the data to a sine-like space and then just use Euclidean distance? That way I can transform my data by hand and then use k-means. Or does it have to be this alternative notion of distance as defined by your inner product? $\endgroup$ – wolfsatthedoor Nov 27 '14 at 14:54
  • $\begingroup$ @robertevansanders You probably don't need this anymore, but you can in fact get a (high-ish-dimensional) Euclidean space representing the triangle wave function by sampling $\{\omega_i\}_{i=1}^m$ according to its Fourier transform (a discrete power law) and building features $z(x) = \begin{bmatrix} \sin(\omega_1 x) & \cos(\omega_1 x) & \dots & \sin(\omega_m x) & \cos(\omega_m x) \end{bmatrix}$ à la Rahimi and Recht (2007). If you want more details, feel free to ask a question and ping me about it here, and I'll write it up. :) $\endgroup$ – Dougal Jun 21 '15 at 8:49
  • $\begingroup$ @robertevansanders ...or you could just map to a 2d space with points on a circle. This won't be exactly the same distance function, but it's much simpler. $\endgroup$ – Dougal Jun 21 '15 at 8:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.