1
$\begingroup$

I'm rather new to Machine Learning but I have been looking into it for a bit now. Specially I've been interested in text classifying solutions and seen how a high level of success has been achieved in classifying words by semantics and extracting meaning.

Right now I'm looking for a way to do something a little more difficult. Suppose that you have a text with plain text and multiple numerical codes in it. Some of these codes can be filtered by ruling, as they have common type characteristics (for example, a date or a telephone number from a specific country). So it should be very easy to classify those numbers by their meaning.

But some other numbers have almost no ruling. Among them we have a user identifier code which can be chosen by the user and can contain numbers, letters or other symbols, which lenghts can vary greatly. The user could provide their information on a written plain text file in a natural (or rather chaotic) way such as this:

Name: Mike
Home address: 234 Port Street
Work addr: Park Lane 14
555-6758
U. code: 334-5
2019/05/01
28 years old

Another participant could send us a text file such as this:

Veronica Phillips
I live in Barcelona, Diagonal Avenue, 6.
Joined 06-23-2019
Phone numer: 973 45 46 47
Personal identity: 87!3
I was associated with class 1A science team when in High School.

So the information provided would require some data to be present (address, name, date joined, user chosen code), and would allow for optional data such as the age or other records. It also would allow poor structure and typos (like in numer insted of number on the last example).

Let's provide one more:

Name:    Addr:          Phone:     Usercode:   Joining date:   More data:
Mark S.  Baker St., 231 987 65 43  777-7.3A     2017-10-10     6.5 feet tall.
         London

As chaotic as it can be (and I'm doing this on purpose for a challenge), a human viewer would easily be able to extract critical information about the user and be able to store their names, addresses, user codes and so on.

Yet here we don't have clear rules for all the strings we want to identify, sometimes code words to identify them won't be just at their side, could be missing, far away (last example) or worded in another way (Usercode / User number / My entry code).

What would be the best strategy to tackle something like this problem using Machine Learning? Is there a good solution for this yet?

$\endgroup$

1 Answer 1

1
$\begingroup$

In formal language theory and computer science, a regular expression is a sequence of characters that define a search pattern. Patterns can vary in complexity.

A simple regular expression MM/DD/YYYY, where all characters except '/' are numeric. This regular expression is of course meant to match thusly expressed dates. With such an example in mind, one should be able to work out what kind of regular expressions to formulate for postcodes, license plates, social security number, etc.

Here is a link to the Wikipedia page that deals with regular expressions: https://en.wikipedia.org/wiki/Regular_expression.

$\endgroup$
4
  • $\begingroup$ That seems unhelpful. The whole problem OP has is that there is no obvious way to find a good Regex for some fields ahead of time, since they are too ambiguous. $\endgroup$
    – jkm
    Jan 3, 2020 at 13:23
  • $\begingroup$ OP can work with a finite collection of regular expressions because, given a country, there are only so many ways a typical individual would format a field of interest. $\endgroup$ Jan 3, 2020 at 17:10
  • $\begingroup$ Finite =/= manageably small. A billion unique patterns is a finite collection. And that's assuming you can think of them all before you implement them, which, in practice, you won't. $\endgroup$
    – jkm
    Jan 4, 2020 at 16:38
  • $\begingroup$ The combinatorial assumption is warranted for such a specific problem if one contemplates separate searches for distinct fields. $\endgroup$ Jan 4, 2020 at 16:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.