Just wondering if anyone is familiar with clustering nominal inputs. I've been looking at SOM as a solution but apparently it only works with numerical features. Are there any extensions for categorical features? Specifically I was wondering about 'Days of the Week' as a possible features. Of course it is possible to convert it into a numerical feature (i.e. Mon - Sun corresponding to nos 1-7) however then the Euclidean distance between Sun and Mon (1&7) would not be the same as the distance from Mon to Tues (1&2). Any suggestions or ideas would be much appreciated.
- Anybody can ask a question
- Anybody can answer
- The best answers are voted up and rise to the top
Commonly nominal variables are dummy coded when used in SOM (e.g., one variable for with a 1 for Monday 0 for not Monday, another for Tuesday, etc.).
You can incorporate additional information by creating combined categories of adjacent days. For example: Monday&Tuesday, Tuesday&Wednesday, etc. However, if your data relates to human behaviour it is often more useful to use Weekday and Weekend as categories.
For nominal variables, the typical encoding in a neural network or electrical engineering context is called "one-hot" -- a vector of all 0s, with one 1 in the appropriate position for the value for the variable. For the days of the week, for example, there are seven days, so your one-hot vectors would be of length seven. Then Monday would be represented as [1 0 0 0 0 0 0], Tuesday as [0 1 0 0 0 0 0], etc.
As Tim hinted, this approach can be generalized easily to encompass arbitrary boolean feature vectors, where each position in the vector corresponds to a feature of interest in your data, and the position is set to 1 or 0 to indicate the presence or absence of that feature.
Once you have binary vectors, the Hamming distance becomes a natural metric, though Euclidean distance is used as well. For one-hot binary vectors, the SOM (or other function approximator) will naturally interpolate between 0 and 1 for each vector position. In this case, these vectors are often treated as the parameters of a Boltzmann or softmax distribution over the space of the nominal variable ; this treatment gives a way to use the vectors in some sort of KL divergence scenario as well.
Cyclic variables are much trickier. As Arthur said in the comments, you'd need to define a distance metric yourself that incorporates the cyclic nature of the variable.
The most logical way to transform hour is into two variables that swing back and forth out of sync. Imagine the position of the end of the hour hand of a 24-hour clock. The
You need both variables or the proper movement through time is lost. This is due to the fact that the derivative of either sin or cos changes in time whereas the
Finally, consider whether it is worthwhile to add a third feature to trace linear time, which can be constructed as hours (or minutes or seconds) from the start of the first record or a Unix time stamp or something similar. These three features then provide proxies for both the cyclic and linear progression of time e.g. you can pull out cyclic phenomena like sleep cycles in people's movement and also linear growth like population vs. time.
Example of if being accomplished:
Now let's try it out:
You can just barely see that there are some after midnight times included with the before midnight green cluster. Now let's reduce the number of clusters and show that before and after midnight can be connected in a single cluster in more detail:
See how the blue cluster contains times that are from before and after midnight that are clustered together in the same cluster...
You can do this for time, or day of week, or week of month, or day of month, or season, or anything.
Assuming day of week (dow) goes from [0, 6], instead of projecting data onto a circle another option is to use:
To understand why, consider the dow as a clock
diff between 6 and 1 could be 6 - 1 = 5 (going clockwise from 1 to 6) or 7 - (6 - 1) = 2. Taking min of both options should do the trick.
In general you can use:
I have successfully encoded Days of the week (and Months of the year) as tuple of (cos,sin) as whuber highlighted in his comment. Than used Euclidean distance.
This is an example of code in r:
Euclidean distance between 0 and 6 is equal to 0 and 1.