This is the problem I am trying to solve: I am looking at user behavior for an online newspaper. I have a behavior period and then I look at the users who bought a subscription within two weeks after the behavior period.

I am trying to find out if the number of distinct categories a user have visited in the behavior period has an effect on the probability of buying a subscription. There are approximately 25 different categories. As expected, we have already seen that there is a correlation between the number of page views (the number of articles an user have visited) and the probability of subscription purchase. In addition, the correlation between number of page views and number of distinct categories is 0.82.

The distributions of both page views and number of distinct categories are strongly decreasing as the values increase. That is, a large share of users have only one or a few page views and have seen just a few different categories.
My question is therefore, how I can find out if there is a relationship between number of distinct categories and the probability of subscription purchase. I remember something about marginalization from when I was a student, but it’s a long time ago. I hope someone is able to explain how I can go about for doing this for the example above.

I'm not sure if it is relevant, but I am mostly using R, but I am also comfortable using Python.


1 Answer 1


The problem that you are facing is that if two predictors are correlated it is difficult to talk about the effect of each over and above the effect of the other which is what you might get if you put both into a multiple regression. This is not a problem with the statistics or the programming, most decent modern software will handle very highly correlated predictors, it is more a problem of logic. I gave a rather extreme toy example in my answer to this How can I have a significant overall F-test but any significant P values for the individual coefficients? Graphical intuition, please? question.

If I were facing this problem I would include both but be careful about how I interpreted the results. I would also be sure to look at the regression diagnostics in case you have any individuals who are extreme on both predictors and exerting undue influence on the model.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.