0
$\begingroup$

I'm looking to extract ids from the body of an email. The ids are similar to flight booking IDs. For example, in an email, I would like to obtain the booking reference (something like MNFF3RGC or MNF-F3RGC or MNFF/3RGC).

Those IDs can come in a great variety of formats and can have separators like "-/_" at different places. The IDs can contain slight formatting differences, so regular expressions are not the best solution as they would need to be updated every time an exception is detected.

I'm thinking of using something like word2vec to extract features of those tokens, and since I have a big set of valid tokens, train a classifier like SVM.

Is this a correct use of word2vec? The examples I'm seeing are related to document classification and not directly word classification. What could be a good approach to the token classification/extraction problem?

Maybe related to this question.

$\endgroup$
1
$\begingroup$

This is advice comes from my experiences building software.


The IDs can contain slight formatting differences, so regular expressions are not the best solution as they would need to be updated every time an exception is detected.

Then build more robust regex. Regex is incredibly powerful and useful (even use something like the "or" pattern: r"(pattern1)|(pattern2)|(pattern3)|...").

Starting with a ML model is going to push the project way out of scope and cause delays. Maybe start reading https://developers.google.com/machine-learning/rules-of-ml/ - its first rule is:

Rule #1: Don’t be afraid to launch a product without machine learning.

$\endgroup$
1
  • $\begingroup$ Thank you for the advice! But what I want to do is to remove the "manual" regex creation and validation, do you know anything to help to remove that? I'm testing regex.inginf.units.it right now. I thought that this would be very simple (train an "id" detector with a good feature extractor), but this seems to be a complex problem after all. $\endgroup$ – h3ct0r Jul 16 '18 at 0:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.