I have built a web scraper to extract data from a site so that it can later be translated/transliterated. I traversed every branch of the HTML tree to extract every string and its raw path along with its tag, class and id. Then operated upon the above data to group them by their x-path(raw path) to obtain the following:

    x-path                                                       members
0   [document]\body\div\a\                                       {X}
1   [document]\body\div\div\                                   {    You will soon be redirected to our partners's site, .    ,     (In case your browser is blocking pop-ups, , .)    }
2   [document]\body\div\div\a\                                 {click here}
3   [document]\body\div\div\div\                               {, }
4   [document]\body\div\div\div\a\                             {PVR: Shalimar Bagh, Movietime: Raja Garden, Satyam Cineplexes: Janak Place (Now INOX), PROCEED, PROCEED, OKAY}
5   [document]\body\div\div\div\div\                           {SIGN IN, SIGN UP, FORGOT PASSWORD, SIGN IN, RESEND CONFIRMATION, SUBSCRIBE TO NEWSLETTERS, Corporate Booking}
6   [document]\body\div\div\div\div\button\                    {Close, Close}
7   [document]\body\div\div\div\div\button\span\               {All}
8   [document]\body\div\div\div\div\div\                       { “I have a good faith belief that the use of the material in the manner complained of is not authorized by the copyright owner, its agent, or the law.†AND “The information in this notification is accurate, and I am the owner, or an agent authorized to act on behalf of the owner, of an exclusive right that is allegedly infringed.â€, FILLING FAST, Select delivery type, Movie Trailers}
9   [document]\body\div\div\div\div\div\aside\a\div\           {SIGN UP NOW, SIGN UP NOW}
10  [document]\body\div\div\div\div\div\aside\button\          {BACK HOME}
11  [document]\body\div\div\div\div\div\aside\div\             {Still haven't , Be a part of an , Still haven't }
12  [document]\body\div\div\div\div\div\aside\div\a\div\div\    {FACEBOOK, GOOGLE, FACEBOOK, GOOGLE, FACEBOOK, GOOGLE}
13  [document]\body\div\div\div\div\div\aside\div\div\a\div\    {SIGN IN, SIGN UP}
14  [document]\body\div\div\div\div\div\aside\div\div\a\span\   {FORGOT PASSWORD?, FORGOT PASSWORD?}
15  [document]\body\div\div\div\div\div\aside\div\form\a\div\   {SIGN IN, SIGN UP, SEND INSTRUCTIONS, SIGN IN, SUBSCRIBE NOW}

In the above data, In column "members" , all strings with the same x-path are grouped together, separated by ",".

Now I want to label these to specify whether they were a part of a Menu, or were buttons, or a static string or links.

How do I do this? provided I don't have a trained data set to predict the labels of the current data based on it.How can I create my own tags for labeling similar to NLTK POS tags?( if am thinking in the right direction)

I studied about clustering, and could relate to Hierarchical Clustering a little. However i still have no clue how to start. Since there are thousands of such rows, I didn't consider labeling of the records manually. Please help, I am pretty new to machine learning.


Don't expect magic to happen.

You will need to provide your objective in a way (training data, math function) that the computer can understand.

There are probably infinitely many 'bad' answers to your vague question, so the chances that some algorithm finds a bad answer is really high, unless you provide a better problem definition.

Often the best way to make your problem more precise is to obtain labeled training examples.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.