I want to create an ML program that cleans up noisy data. I have the raw text features and the labels. ML programs tend to prefer numeric data, so I need to encode my text features. This is an example of how my data looks like:
**Raw text feature** **Label** 10/01/2017-XYZ123456-JOHN DOE - ABC2345 Doe, John 01 02 14 - J A N E D O E _(temp) QWERTY12 Doe, Jane 01feb2018#02mar2018#john#smith#ID12345 Smith, John
My first step is to find the right encoder. I do not want a one-hot encoder because there is a large, finite and undefined set categories/Labels. I need a more dynamic encoding that can vary in size and works great with data it has not seen before. Question 1: Any suggestions on how to encode this data?
Then I have to choose an ML classification method. I would go with neural networks because of the complexity of the feature. The text feature is a sequence of characters which I am hoping my ML program would be able to predict (or extract) a name in the form [Last Name, First Name]. I would use a Recurrent Neural Network, for which I will figure out the optimal hyperparameters once I encode the text data properly.
Question 2: Any suggestions on an alternative to a recurrent neural network? Maybe an approach outside neural networks?
Thanks for your help!