I have a list of items that represent cells, now i query each cell value to get a set of possible categories that this cell might belong to, these categories are weighted. For Example:

Microsoft is: {Software=83.543266, Video Game Platform=132.297455, Degree=71.624733, Organization=208.17038, Programming Language=66.998901, File Format=67.159828}

apple is: {Physicist=50.008327, Organism Classification=145.239532, Video Game Platform=86.5653, Computer=60.457172, Organization=199.53299, Record label=53.477039}

Banana is: {Musical Group=27.214769, Organism Classification=136.063583, Writer=27.566528, Musical instrument=29.285137, Organization=36.962833, TV Program=32.693085}

Blackberry is: {Software=23.69421, Organism Classification=64.572891, Operating System=90.257011, City/Town/Village=16.836142, Computer=34.484653}

The weights that i get for each result are independent at cell level, so Microsoft having a very high confidence as an organization does not relate to the confidence score of Apple organization confidence.

Now i have two problems, the first is to compare two different cells and compute a similarity score, for example how similar apple is to Microsoft having these possible categories for each.

Second, i want to deduce a global common category for the column that represents these cells, for example if i run a simple algorithm that will calculate the number of categories occurrences and the average of their confidences will not be accurate.

I will appreciate any help.

  • $\begingroup$ What would the point of the similarity score be? For example, it's entirely sure that "Sun" is a large IT corporation, just as Microsoft is, but it's surely likely to have a much higher score for "celestial object", so your similarity may be quite low... $\endgroup$ – naught101 Jun 7 '12 at 2:35

For the first question, a cosine of the angle between the two (aligned) category vectors of two given items is often used as a measure of similarity.

I don't quite understand the second question. Are you referring to some sort of dimensionality reduction / latent factor approach applied to the category vectors?

  • 1
    $\begingroup$ This is something like what I was thinking too. Probably should add something about the computation of cosine ($cos\theta=\frac {a\cdot b} {\left\|a\right\|\left\|b\right\|} $)... $\endgroup$ – naught101 Jun 7 '12 at 2:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.