0
$\begingroup$

I have a set of strings of dimension $10,000$. I want to group similar strings together in one group, perform clustering. As string metric, I am using the Levenshtein distance.

Simply, with the Levenshtein distance I'll just compute distance between $2$ strings and then by using a threshold the clustering algorithm will make the decision if they can be grouped or not. This is not enough. I am looking for a special measure to study the relation between the strings.

For example: door and entrance wont be grouped together if I just compute the Levenshtein distance, in fact there is nothing in common between these $2$ words. But logically they are connected and can be grouped together, since door and entrance are basically the same.

  • Have you ever come across such a problem?
  • Is it similar to the Semantic similarity measure?
|cite|improve this question
$\endgroup$
1
$\begingroup$

Have a look at WordNet. Quoting from http://wordnet.princeton.edu/:

WordNet is a large lexical database of English. Nouns, verbs, adjectives and adverbs are grouped into sets of cognitive synonyms (synsets), each expressing a distinct concept. Synsets are interlinked by means of conceptual-semantic and lexical relations. The resulting network of meaningfully related words and concepts can be navigated with the browser. WordNet is also freely and publicly available for download.

|cite|improve this answer
$\endgroup$
  • $\begingroup$ yes I was just looking @micans. I found the wordnet website too wordnet.princeton.edu/wordnet/download/current-version I wanted to make sure that I am going in the right direction. What I am looking for is semantic similarity right? $\endgroup$ – Hani Gotc Mar 11 '14 at 11:27
  • $\begingroup$ Semantic similarity does not seem to be a very well defined technical concept, more a loose general term, so I would not worry about 'doing it correctly'. The most important thing is what you want to get from your data. Certainly, WordNet is a highly useful resource and will help you with meaning of words. It is a good place to start. Regarding 'Semantic similarity' (e.g. the wikipedia page), I would just use it to gain some background knowledge and/or ideas for future reference. The crucial aspect to all this is your data and what you hope to get from it though. $\endgroup$ – micans Mar 11 '14 at 11:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.