Multi-label classification: Predict product category

Question

I want to predict to which product category a product belongs. A total of 400k products need to be translated from the old (less refined) to the new product category tree. (E.g. alarm clock used to fall under 'Electronics' and will now belong to 'Alarm clocks'.) So far 36k products have already been partly allocated to ~400 (out of 800) new product categories. The filling rate ranges from 1% to 95%.

Product data (among others) contains variables: name, description, price, dimensions, color and the old label . The idea was to construct features out of the unstructured variables through tokenisation -> TF-IDF.

Proposed Approach:

1. Train one multi-label prediction model (e.g. Ridge classification + stratified CV) on the labeled data. Then predict the category only for subset that, based on the old product tree, contains all possible products. (e.g. predict if unlabelled 'Electronics' products are 'Alarm clocks')
2. Based on the predicted probability present the unlabelled product to a content manager that, if labelled, would result in the highest information gain.
3. Propose to which extend the remaining 400 categories should be filled (e.g. 60%) and which products to label first.

What would your preferred approach be?

Answer 1

Since you have ~800 categories as the classification variable, in my understanding, the accuracy of the classification can be increased by better models than ridge regression model alone. Neural networks with multi layers can be more adept and also you can build an ensemble of models to arrive at the final classification.

The text data can also be used to group together based on association metrics to arrive at a class variable based on association of text. Another variable can be the clustered variable containing products which can be clustered together. These two pieces of information can help the final model delineate the products much better before assigning them to a particular category. Hope it helps and all the best :)