I am trying to match new product description with the existing ones. Product description looks like this: Panasonic DMC-FX07EB digital camera silver. These are steps to be performed:

  1. Tokenize description in form of: Panasonic => Brand, DMC-FX07EB => Model, etc.
  2. Get few candidates with similar features
  3. Get the best candidate.

I am having problem with the first step (1). In order to get 'Panasonic => Brand', DMC-FX07EB => Model, silver => color, I need to have index where each token of the product description correspond to certain attribute name (Brand, model, color, etc.) in the existing database. The problem is that in my database product descriptions are presented as one atomic attribute e.g. 'description' (no separated product attributes).

Any suggestions how can I tokenize descriptions and build index (inverted index) where each token corresponds to certain attribute?


The best way to build index of product attributes is to use some APIs for product specifications: I am trying with one from Bestbuy.com, but attributes are not defined in a proper way. Any suggestions for the Product Specifications APIs? or similar solutions?

So far https://www.semantics3.com/ seems the best solution.


So the first problem is that you need to build a brand dictionary, yes? In most retail representations of product titles, brands are either the first 1-gram, 2-gram or 3-gram in the description OR everything following the word "by" at the end of the description, such as "hdtv by Samsung". So extract out these features based on token position and build an entropy measure for discerning information that is provided by words in addition to the first word, second word, etc. , such as "Sony Playstation" or "black and decker". Get something good enough or else you will drive yourself mad trying to boil the ocean, especially if you attempt to deal with accessories.

| cite | improve this answer | |
  • $\begingroup$ Thank you, but I think that you misunderstood what I want to achieve. It is true that I need Brand, but I also want to extract other attributes such e.g. Model, Color, Type (4,4s,5s) etc. $\endgroup$ – dzeno Nov 25 '14 at 11:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.