2
$\begingroup$

I have the following situation. Using scopus (https://www.scopus.com) I was looking for scientific articles pertaning to a concrete domain. I obtained 5.000 results. Each entry contains a lot of information about the article (around 40 different features) like country of the authors, research domain, year, number of citations, funding achieved, etc. I wanted them to test some web tool we have developed. So we emailed all of them (since each scopus records contains email for the author of each article) and we observed the following:

a) only around 1.000 visited the website b) only around 100 registered on the website c) only 20 used the tool d) only 5 contacted us showing real interest on the tool

Here, we would like to analyze data and understand which are the main features, between all those who finally contacted us. We guess that if we find some pattern, then and next time, we will try to find articles of the people pertaining to that pattern.

Not so clear is for us how could we achieve this. One approach we guess is to statistically analyze data. Here I have always worked with numeric fields but not text, so I do not know what would be the best approach. Or perhaps we could also train a machine learning method, so once trained, we can input it other articles so that it will try to predict the success chances.

So my question here is, what do you think is the best approach to follow.

PS: If somebody can help with this, we are working on a scientific publication, he/she could join as coauthor

Improve this question
$\endgroup$

1 Answer 1

Reset to default
5
+50
$\begingroup$

  1. The data of who interacted with the mail in what way is going to contain a lot of noise. For example you have no data on how busy the recipients were at the moment they received the mail, while this most likely has a massive effect on the response rate. Also things like local-time and how regularly they check that specific e-mail address is going to have a big effect. This is on top of general unpredictability in people.

  2. Especially considering my first point, 5 or 20 datapoints of positive examples is going to be far to few to draw any concrete conclusions. You might be able to analyse the 100 people who registered, but I would be surprised if it would lead to repeatable results. In my opinion, you can call yourself lucky if you get meaningful results from analyzing the 1000 people who visited your website versus the 4000 who didn't.

  3. In general choose to work with structured data over unstructured data like text if it is at all possible. There are ways to extract features from text, but since text is ambiguous and hard to interpret, there is going to be a lot of noise in those features. That noise is going to go on top of the noise mentioned in point 1. Especially if you already have 40 features, why not use those instead of the text? You can change categorical data like country names into numbers by changing it into a vector. For example suppose there are 193 possible countries, you can change a given country into a 193 length vector. This vector would have 192 zeros and a single 1. The position of the 1 would represent the country. So a 1 on position 54 might represent France and a 1 on position 167 might represent Russia.

  4. Instead of finding people who are more likely to respond, you are probably far more likely to increase your success rate by changing your e-mails and sign-up process. Try 5 different e-mails and sent each person randomly one of the 5. Then compare response rates between them. Also, if 90% of the people who clicked the link didn't sign up, ask yourself: do I need a sign-up page? If you really do, is there anything you can do to make signing up easier? Each click you save your respondents is going to have a positive effect on your response rate.

Improve this answer
$\endgroup$
4
  • $\begingroup$ thanks for your really useful comments. I agree with all your points. I am just worried a bit when you tell the difficulties of mapping text to numbers. are not there established approaches for this? $\endgroup$
    – Open the way
    Commented Oct 26, 2016 at 19:48
  • $\begingroup$ and another thing, if one focuses just on the main 5 or 20 datapoints, I see that the dataset will be unbalanced, but I have read that methods such as random forest can work here, what do you say? $\endgroup$
    – Open the way
    Commented Oct 26, 2016 at 19:49
  • $\begingroup$ There is a difference between mapping an article to text and mapping categorical data (e.g. a country name) to numbers. The former is much more difficult then the latter. For texts: there are many different ways to say the same thing and very similar ways to say very different things (imagine the effect the word 'not' can have on a sentence). This makes analyzing text very difficult. For categorical data, you can just take a vector the same length as the number of options (for countries that would be roughly 200) and then let each option be a 1 on a specific spot. $\endgroup$
    – dimpol
    Commented Oct 31, 2016 at 8:56
  • $\begingroup$ The problem of unbalanced datasets are for example when you have 1800 examples of category A and 300 of category B. Here just guessing A all the time will give you almost 86% accuracy on the training set. There are various ways to deal with unbalanced datasets. However in your case I strongly suspect you just have not enough training examples. This is going to be a fundamental problem for every machine learning algorithm, you most likely just don't have the information to show any patterns. $\endgroup$
    – dimpol
    Commented Oct 31, 2016 at 8:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.