What sort of (supervised?) leaning algorithm should I be using?

Question

I am selecting manually several hundreds Google Alerts (GA) texts those that are indeed relevant for my research vs those which are not (despite they are all triggered by some relevant search keywords).

Basically each week I get several hundreds GA email such as:

https://www.dropbox.com/s/u7rp0ez1tamq001/Alerte%20Google%C2%A0-%20laitier%20-%20lucam1968%40gmail.com%20-%20Gmail.pdf?dl=0

and

https://www.dropbox.com/s/1ubx5enw6tc90hj/Google%20Alert%20-%20latte%20-%20lucam1968%40gmail.com%20-%20Gmail.pdf?dl=0

From such emails I create a file such as:

https://docs.google.com/spreadsheets/d/1ZZfwUIxZ2WxDARMej1kKCt1cR3TPQwIbSJkK-vsc2vA/edit?usp=sharing

This is really becoming a time consuming procedure, hence my decision to try applying artificial intelligence solutions to such a case.

What sort of supervised learning algorithm would you advise that I adopt so that it can learn by example from my choices and decide on my behalf whether to retain the piece of information or not - see Retain=Yes/No in the attached file. That is: I can classify a few hundreds cases and then let the algorithm learn and classify future/additional data. I plan to regularly review such a classification, correct missclassifications and train the algorithm again with the objective to improve its ability to correctly classify the GA texts according to the examples I provide the algorithm.

I am also looking for solutions in software. My preferred software is R, but I am also open to other software.

Answer 1

First, I think supervised learning may not good for what you want to do, because of limited data.

I can classify a few hundreds cases and then let the algorithm learn and classify future/additional data.

Few hundreds is very likely to be too few to build a system has reasonable performance. For example, this spam data set in UCI has ~ 4600 examples.

From what you described, a rule based system may work better. You may try to summarize some logics you used to classify if an alert is retained or not. With limited data, a rule based system usually would work better.

---

If you want to use supervised learning:

You are missing a crucial step for supervised learning: feature extraction. The data you presented is very raw. How can we feed a model with such data? are we feeding in plan text or something else?

If you can write same parser to extract useful information, such as alert time, alert title etc, that would be better than plan text file. With extracted features, almost all the supervised learning algorithm can be used.

Take a look at the SPAM data set I linked above, there are $57$ features people derived. Deriving these features is your missing step.