I have 1 short string of text (let's say it's a tweet, max 140 characters):
"A review of my beloved Roku 3 media player"

I also have a larger body of text (like a blog article, hundreds of words) which I know is related to the tweet:
"The Roku 3 media player is a great way to watch your favorite ...."

The tweet and blog article are both about the Roku 3 media player specifically. Same author, and they share many of the same phrases, words, collocations, etc. It's likely that the string "Roku 3" appears in the text, along with variations like ""Roku 3 streaming media player", "Roku 3 player" etc

I then have 10 other tweets, some which are related to the "Roku 3 media player", some of which are not (but very similar):

RELATED "A good review of the Roku 3 media player"
UNRELATED "The Roku 2 media player review"
RELATED "Roku 3 is amazing"
RELATED "The Roku 3 is better than the Roku 2 by far"
RELATED "The Roku version 3 streaming media player, fully reviewed"
UNRELATED "A comparison review of the top 3 media player boxes. Roku, Android, Toshiba"
RELATED "Roku 3 streaming media player reviewed"

Those are some examples, and I would have 10 in total. All of the tweets contain "Roku", some are about "Roku 2" and are unrelated, one uses "Roku version 3" and is related etc. Obviously, this is a very small data set.

What is the best method to classify each of the 10 tweets as relevant or not, in relation to the first tweet and blog article? What sort of features would be useful?

  • $\begingroup$ Just on a purely theoretical basis, would it not be possible to be exclusionary for some of the results? As in, exclude Roku results where Roku 2 is mentioned Roku 3 is not, but include results with both Roku 2 and Roku 3 mentioned. However, I think on a broad, automated scale this is a very difficult task. I think this explains why so many search engines have difficulty with the concept of 'relevance'. I'd love to hear your feedback on this suggestion. $\endgroup$
    – user
    Oct 16 '13 at 5:56
  • $\begingroup$ I think that part of the reason that this may be difficult is that you would need sets of related data. Using automated methods, I would think it would be quite difficult for an engine to determine that that are indeed two different Roku, 2 and 3 (and perhaps the original). $\endgroup$
    – user
    Oct 16 '13 at 6:01

On a data set this size, it can be pretty tough to learn anything at all; as a baseline, it might be worth it to learn the N-Grams shared by the two examples, and run from there. For a training pair consisting of one short string and a longer article, it would be relatively easy to find the common N-Grams between the article and tweet, and then analyze the frequency with which those N-Grams appear in the test data. In the case you presented, the important N-Grams seem to be "Roku 3" and "Roku 3 Media Player" (a bigram and 4-gram, respectively), both of which are shared by the training data strings.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.