Take the 2-minute tour ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

Just wondering if anyone is familiar with clustering nominal inputs. I've been looking at SOM as a solution but apparently it only works with numerical features. Are there any extensions for categorical features? Specifically I was wondering about 'Days of the Week' as a possible features. Of course it is possible to convert it into a numerical feature (i.e. Mon - Sun corresponding to nos 1-7) however then the Euclidean distance between Sun and Mon (1&7) would not be the same as the distance from Mon to Tues (1&2). Any suggestions or ideas would be much appreciated.

share|improve this question
    
(+1) a very interesting question –  steffen Jul 19 '11 at 9:22
1  
Cyclic variables are best thought of as elements of the unit circle in the Complex plane. Thus, it would be natural to map the days of the week to (say) the points $\exp(2 j \pi i / 7)$, $j=0, \ldots, 6$; i.e., $(\cos(0), \sin(0))$, $(\cos(2 \pi/7), \sin(2\pi/7))$, ... $(\cos(12\pi/7),\sin(12\pi/7))$. –  whuber Jul 19 '11 at 14:02
1  
would i have to code up my own distance matrix then specific to cyclic variables? just wondering if there were already existing algorithms for this type of clustering. thx –  Michael Jul 20 '11 at 3:48
    
@Michael: I believe you will want to specify your own distance metric that is appropriate for your application, and that is defined over all the dimensions in your data, not just the DOW. Formally, letting x, y denote points in your data space, you need to define a metric function d(x,y) with the usual properties: d(x,x)=0, d(x,y)=d(y,x), and d(x,z) <= d(x,y)+d(y,z). Once you've done that, creating the SOM is mechanical. The creative challenge is to define d() in a way that captures the notion of "similarity" appropriate to your application. –  Arthur Small Dec 14 '12 at 14:48
add comment

2 Answers

Commonly nominal variables are dummy coded when used in SOM (e.g., one variable for with a 1 for Monday 0 for not Monday, another for Tuesday, etc.).

You can incorporate additional information by creating combined categories of adjacent days. For example: Monday&Tuesday, Tuesday&Wednesday, etc. However, if your data relates to human behaviour it is often more useful to use Weekday and Weekend as categories.

share|improve this answer
add comment

For nominal variables, the typical encoding in a neural network or electrical engineering context is called "one-hot" -- a vector of all 0s, with one 1 in the appropriate position for the value for the variable. For the days of the week, for example, there are seven days, so your one-hot vectors would be of length seven. Then Monday would be represented as [1 0 0 0 0 0 0], Tuesday as [0 1 0 0 0 0 0], etc.

As Tim hinted, this approach can be generalized easily to encompass arbitrary boolean feature vectors, where each position in the vector corresponds to a feature of interest in your data, and the position is set to 1 or 0 to indicate the presence or absence of that feature.

Once you have binary vectors, the Hamming distance becomes a natural metric, though Euclidean distance is used as well. For one-hot binary vectors, the SOM (or other function approximator) will naturally interpolate between 0 and 1 for each vector position. In this case, these vectors are often treated as the parameters of a Boltzmann or softmax distribution over the space of the nominal variable ; this treatment gives a way to use the vectors in some sort of KL divergence scenario as well.

Cyclic variables are much trickier. As Arthur said in the comments, you'd need to define a distance metric yourself that incorporates the cyclic nature of the variable.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.