5
$\begingroup$

For my work, im using the multilabel dataset from this webpage. Few dataset which are listed in the page (for, e.g bibtex) have nominal attributes, i.e attribute values are 0 and 1.

My queries are given below

  1. Is it valid to run kMeans clustering algorithm on these nominal dataset to get centers and target label which are meaningful?

  2. Otherwise, to run kMeans algorithm (forget abt the target label), i need to convert this nominal dataset into numerical dataset. What is the standard procedure of doing it. I can normalize each instance, but it just gives me a real number with equal value for an instance.

  3. I would also like to reduce the dimension of nominal dataset such as rcv1v2. How do i go about it. I can use any Feature selection technique but it requires an optimization criteria. But in my case, i need to compare the result of different algorithm on this dataset which have different optimization criteria, so i got into trouble of choosing which criteria. Is there any technique of selecting a top features?

$\endgroup$
6
$\begingroup$

Although formally you may do K-means clustering on nominal data after converting nominal variables into dummy variables, this is regarded inadequate approach. To use K-means meaningfully, you must have all variables at scale (interval or ratio) level.

One of the ways to quantify a set of nominal variables is to apply multiple correspondence analysis. It can be seen as a dimension-reduction technique, like PCA, only for nominal data. You could use the resultant quantifications (the coordinates) as the input to K-means, if you like.

$\endgroup$
  • 1
    $\begingroup$ I tried to get materials for MCA by googling. But didnt find anything much on it. By any chance do u know any link where i could find much materials on the same so that i can able to convert nominal data into coordinates as you mentioned for k-means and such algorithms. $\endgroup$ – Learner Mar 12 '12 at 5:10
1
$\begingroup$

For nominal data that represents token occurrences in a text document as in RCV1v2 you can use the TF-IDF transform as a way to normalize the data.

$\endgroup$
  • $\begingroup$ is it ok to apply TF-IDF transform of token occurrence matrix? We usually apply them to word occurrence matrix which contains the frequency of word occurring in the documents. $\endgroup$ – Learner Mar 12 '12 at 5:06
  • $\begingroup$ I would say it is ok. Just try and see if it improves the cross validated score (e.g. f1-score). $\endgroup$ – ogrisel Mar 12 '12 at 8:18
1
$\begingroup$

I'd calculate a similarity matrix using jaccard distance and then run k-means

$\endgroup$
  • $\begingroup$ I dont want to calculate similarity matrix. I need only feature matrix with numerical values, since i need to run Non-negative matrix factorization later which requires feature matrix. $\endgroup$ – Learner Mar 12 '12 at 4:45
1
$\begingroup$

k-means is not appropriate for this kind of data, for a very simple reason.

Assume you convert the data set so that value a = 1, value b = 2, value c = 3, a cluster that consists of 10 objects with label a and 10 objects with label c will have a center at the value of b. And even when you have binary data, what is an average of 0.1314 supposed to mean?

Do not convert the data set. Choose an appropriate distance function that can handle binary attributes, and use an algorithm that only needs distances and not a vector space. Methods that require an euclidean vectors space (such as k-means, in order to compute a mean) are not meaningful for that that just isn't from an euclidean space.

$\endgroup$
  • $\begingroup$ Your idea is ok but i need to compare various algorithm which needs feature matrix. $\endgroup$ – Learner Mar 12 '12 at 10:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.