4
$\begingroup$

Combination of categories is not possible as each class is a distinct brand.

One similar challenge was classifying objects using images (link below) but I could get any specific direction from few papers published regarding the same http://www.image-net.org/challenges/LSVRC/2014/

Alternative approach if algo is not suitable is to build Wordnet/frequency based structure out of available brand-word data & create distance matrix from it Another one is to aggregate the categories & build 2 levels of brand prediction (not sure if it will be effective at all)

Improve this question
$\endgroup$
7
  • $\begingroup$ I am not sure I understand your problem, how big is your dataset ? Why is the number of clusters a problem ? (It is a naive question as I've never thought it could be a problem) $\endgroup$
    – Riff
    Commented Jun 22, 2016 at 7:32
  • $\begingroup$ My data set is about 100K rows with product information(scraped from web) & high level product category (about 500 distinct categories) with product brands (about 35K) which I want to predict $\endgroup$
    – wololo
    Commented Jun 22, 2016 at 8:13
  • 1
    $\begingroup$ + Not a clustering problem $\endgroup$
    – wololo
    Commented Jun 22, 2016 at 8:14
  • $\begingroup$ Well if you have no separation problem and all your classes are somewhat balanced in your training dataset then I don't see why a simple GLM with variables selection wouldn't work (though it will take some time to fit it considering the dataset). There's also random forests but they are doing worse than GLM when trying to predict outside of their 'training ranges' so I would stick to GLM $\endgroup$
    – Riff
    Commented Jun 22, 2016 at 9:53
  • $\begingroup$ Clustering and classification aren't that fundamentally different in my opinion, both tries to group similar individuals in clusters whether or not you actually have a prior insight on the clusters (supervised or unsupervised). The means and approach are different but not the endpoint, I have seen uses of k-means clusters for further 'predictions'. $\endgroup$
    – Riff
    Commented Jun 22, 2016 at 9:55

1 Answer 1

Reset to default
2
$\begingroup$

With the limited information that you've given about the dataset, here are few questions that you can consider. I believe you'll need to frame your problem better and purely classification algorithm A Vs. B is not the issue.

Things to Consider

  • Do you fundamentally believe that the information/independent columns can predict your labels?

  • Do you really need the classification to accurately work for all kind of brand labels?

    • For certain top level brands you can derive the correlation (if product categories are sportswear and sport equipment it is likely to be a Nike). But for certain brands where # of products are small, deriving such correlation may not be possible.

  • The problem looks ill posed due to size of dataset (100k rows Vs. 35k labels). A large number of your categories are going to have a single data point where most of the methods are going to fail (for example, it will be hard to do cross-validation on many categories).

Suggestion

  • Re-frame your problem. Check if you really want to predict 35k possible categories or you are better off considering, let us say, top 100 brands which contribute most of the products.
  • For such large scale categorization problems, you can start with neighborhood based models.
  • Check the discussion discussion forum of this wikipedia categorization challenge on Kaggle for more ideas. [1]
Improve this answer
$\endgroup$
0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.