I have components basically divided into two main categories. AWS and Azure. For eg:

     AWS               Azure
 AWS Sagemaker .      Azure ML Service
 Amazon Rekognition . Cognitive sevices
 Amazon CloudFront .  Azure Content Delivery network
 EC2 .                Azure Virtual Machines

The idea is to train a classifier which takes two parameters

1) type (Azure or AWS) and 
2) matching name.

For ex if a user enters CDN with type as Azure we want to return Azure Content Delivery Network and in case of AWS we return Amazon CloudFront. similarly if a user enters instance with Azure we need to return Azure Virtual Machines and EC2 with AWS.

The idea is to train an algorithm which when encounters a phrase will return the closest component in a particular category.

I created a Naive bayes classifier(Not sure if it is right approach) for all such components in Azure and AWS and I have added (instance,EC2) {text,label} format for all the components. My sample data looks like this

 Text                          Type     Label
 Content Delievery Network .   AWS    .Amazon Cloudfront
 Content Delievery Network .   Azure    Azure Content Delivery network
 Content Network           .   Azure    Azure Content Delivery network
 CDN .                     .   Azure    Azure Content Delivery network
 Content Delievery .   AWS    .Amazon Cloudfront

Is there a better way to do this? using Topic Modelling or by training Word2Vec on the documents of these components?


1 Answer 1


Why is this a classification problem? If you know the most common labels (it looks like "azure" is always in Azure labels, and "amazon" or "AWS" in AWS labels), why do you need to train a classifier. I would just convert the label strings to lower case, and then search for aws, azure, amazon, etc., among the most common labels that you know are used for aws vs azure, and then point the user to the correct service. If you had a lot of different labels, then I would look at word frequencies associated with each service, and use that to generate 2 clusters (unsupervised) which is not supervised classification.

Further, if you want to address a classification problem, you need a training dataset, for which you would have to reveal how many records and how many features are used. I would also use k-nearest neighbors (KNN) instead of NBC, since NBC will require quantitative features or at least 4 categories in each feature. Or a lot of features with 2 categories each.

Also, you have qualitative data based on categories, not continuosly-scaled data (numbers), so it limits you to what you can use for classification. If there was a greater variety of words used in you dataset, you could use text mining to assign labels to clusters (i.e., 2) and then see if the word frequencies are predictive of class (cluster). That would be called "duo mining" -- text mining combined with classification.

  • $\begingroup$ i am not familiar with the last approach. DO you have any example ? $\endgroup$
    – Rohit
    Oct 31, 2019 at 17:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.