3
$\begingroup$

The problem I am trying to solve is to find if presence of a particular tags/topics in an article make it viral.(Page views is my metric) The data I have is the article tags and the number of page views it received. My data looks something like this.

ArticleA => [tag1,tag2,tag3,...,tagN] => PageViews(ArticleA)

I have a list of 5000 tagged articles with its corresponding page views, now I would like to find out if presence of certain tags guarantee success of a post(gathers higher page views). Could anyone suggest a simple/straightforward statistical model for me to get started with this exercise?

$\endgroup$
  • 2
    $\begingroup$ How many tags are in the data? $\endgroup$ – user31264 Feb 12 '17 at 17:43
  • 1
    $\begingroup$ Might a close voter explain what they find unclear? The OP has a list of tags for each article, essentially a set of boolean predictive features, and is looking for a simple model to analyze how they predict a (count) predicted feature. This seems clear—voting leave open. $\endgroup$ – Sean Easter Feb 13 '17 at 1:45
  • $\begingroup$ @user31264 Typically there are 3-5 tags on a post, and the tags for an article is always chosen from a set of 40 tags. $\endgroup$ – Varun Feb 13 '17 at 4:32
  • 1
    $\begingroup$ @Varun - if there are only 40 different tags, and 5000 articles, then a plain linear regression would be reasonable. Maybe you should use log(page_views) or log(page_views+1) as your target metrics. $\endgroup$ – user31264 Feb 13 '17 at 9:35
  • $\begingroup$ I rewrote the answer. $\endgroup$ – user31264 Feb 13 '17 at 9:45
2
$\begingroup$

In your case, you do not seem to be looking for accuracy in the prediction, but try to understand which factors are influencing the success of an article.

Regression may be quite hard to start with in your case (since the distribution of the output will need some pre processing). I would try to turn it into a classification problem "Has the video been viral ?" being equivalent to " PageViews(ArticleA) > 10 000" (per example).

Then I would try to calibrate a logistic regression (I would not bother with regularization in a first time - you have relatively few columns compared to the number of observations) or a naive Bayes.

These two methods will help you (looking at the coefficients or the probabilities) to have an idea of the influence of each variable on the "virality" of the article.

Another approach would be a decision tree, with a relatively low depth (to ensure each leaf represents enough information). This can help you identify combination of tags that contribute the most to the "virality" and is quite easy to visualize !

$\endgroup$
3
$\begingroup$

Try some kind of regularized linear regression, like ridge regression or lasso regression.

EDIT:

You have 40 different tags, and 5000 articles. You want to predict the number of page views based on which tags the article has. You ask for a simple/straightforward statistical model to get started with this excercise.

You have 40 binary explanatory variables (tags) and one dependent variable (number of page views).

You cannot have anything simpler than a linear regression. Since the number of explanatory variables (40) is much less than the number of examples (5000), you would have a reasonable performance even without regularization.

Since the number of page views can probably change by orders of magnitude (it is quite possible that one article would receive 1000 times more views than another one), it would be reasonable to use log(page_views) or log(page_views+1) as the dependent variable, rather than just raw page_views.

After you have done this, in order to improve your model you may want to try also 40*39/2=780 combinations of two tags as additional explanatory variables, in which case you might significantly benefit from regularization.

There are many other approaches to the problem, like neural nets, or random forest. But they are far from simple.

$\endgroup$
  • $\begingroup$ This is being automatically flagged as low quality, probably because it is so short. At present it is more of a comment than an answer by our standards. Can you expand on it? You can also turn it into a comment. $\endgroup$ – gung - Reinstate Monica Feb 12 '17 at 19:36
  • $\begingroup$ This answer does not deal with the question that was posed. $\endgroup$ – Michael R. Chernick Feb 12 '17 at 19:42
  • $\begingroup$ This does not provide an answer to the question. To critique or request clarification from an author, leave a comment below their post. - From Review $\endgroup$ – kjetil b halvorsen Feb 12 '17 at 20:04
  • $\begingroup$ @kjetilbhalvorsen - "This does not provide an answer to the question." - yes it does. "To critique or request clarification from an author, leave a comment below their post" - it is neither a critique not a clarification request. You comment make it seem as you hadn't read my answer. There is a question, for TWO DAYS no one proposes any answer whatsoever, and when I finally do, the very same people who failed to answer the question start to criticize my answer and mark it for deletion. $\endgroup$ – user31264 Feb 12 '17 at 21:43
  • $\begingroup$ @gung - it is short because the question does not need a longer answer. There is a question, for TWO DAYS no one proposes any answer whatsoever, and when I finally do, the very same people who failed to answer the question start to criticize my answer and mark it for deletion. $\endgroup$ – user31264 Feb 12 '17 at 21:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.