3
$\begingroup$

I'm working on a project where the goal is to predict sales of certain generic products. There are many features, but social media metrics are causing me currently some headache. Social media metrics are a decent indicator of success. I have data related to Twitter and Facebook. The count of 'likes' seems to be a quite simple but effective feature in this context. However, not all products are marketed on both platforms. ~50% of observations are present on both platforms. ~80% in Facebook and ~60% in Twitter. ~10% are not present on either platform.

The VIF for both variables is <2. The pearson correlation coefficient is 0.68. If I put them into the same OLS regression model, Twitter has negative coefficient, while Facebook remains positive. In separate models both are positive and significant.

What I have tried:

  1. If I remove the NaN rows the model will work only on 50% of cases.
  2. If I filled the NaN values with 0 it creates a huge penalty for the products that are only present in Twitter, as the coefficient is negative if both variables are present in model.
  3. Summing the values is not a solution either, because the Twitter follower numbers tend to be about twice bigger.

What I have not tried:

  1. Use standard scaler and then sum the values?

Current solution:

What I have done so far is creating a new variables social_media_audience_size which is a a linear combination of the two features based on the univariate beta coefficients as multipliers. y = ax_1 + bx_2 a and b are the coefficient of Facebook likes and Twitter followers. However, I'm not sure if this is an appropriate solution to my problem.

Questions:

What could be a generally accepted way to address the specified issue?

Also, does this problem have a name? I'm having hard time finding any sources dealing with this issue, but I don't see why this would be very unique problem. I would also be interested in any of academic references where this problem is addressed!

Finally, I'm not entirely sure if my variables even are multicollinear, considering that the VIF is small? How should I analyze and report multicollinearity in case there is small VIF?

$\endgroup$

2 Answers 2

2
$\begingroup$

First of all, if the VIF for both variables is under 2, then this is not a problem of multicolinearity. Neither is this an issue of "missing data" due to item nonresponse or anything like that. Instead, it sounds like your "problem" is just that the world works in a particular way: you are trying to predict success as a function of "likes" on various social media platforms, but if a product wasn't promoted on (say) twitter than the "likes" variable is (understandably) undefined. This is a not a statistical problem that you can solve with statistical tests but THEORIETICAL problem that you need to solve by thinking about what you are trying to do. Different approaches to this issue reflect different research questions and ways of thinking about what you are trying to do.

If you don't do anything to the missing values but run a model with both "facebook likes" and "twitter likes" in it, then the model will only contain products that were marketed on BOTH platforms, and it will tell you the impact of twitter likes CONTROLING for facebook likes (in other words, what would be the impact of having one more twitter like, holding facebook likes constant). If what you are trying to do is predict the relative importance of twitter or facebook likes, then this actually seems like a good model, because it limits the sample to products that were advertised on both (making it a "fair" comparison) and tries to find the effect of each kind of like, net of the other.

On the other hand if you replace the "NaN" on likes with zero for a product that wasn't marketed on a platform then, this reflects a view that "if you don't market a product on twitter then by definition the numbers of likes you get on it it zero." So yes it "penalizes" companies that don't market on twitter (or facebook) but maybe it should: in real life it is TRUE that you can't get likes on a platform if you don't promote on it right? So if you just want to know the extent to which "likes" predict success, and you want to acknowledge the real world fact that likes require promotion in the first place, then this might actually be a fine model.

In short: what you describe isn't really a "problem" but just the general issue of "how do I set up my data to make sure I'm answering the right research question." And there is no one answer: it depends on what research question you are trying to answer, and your background "theories" about how you are thinking about the various issues. So the solution here is to just think hard about the substantive implications of making different choices here, and choose the approach that most reflects the assumptions you want to make and the question you are trying to answer. I know that's unsatisfying but that's how stats works in practice.

$\endgroup$
5
  • $\begingroup$ About the first point, I've understood that regardless of low VIF, there can be multicollinearity. "The lower precision, switched signs, and a lack of statistical significance are typical problems associated with multicollinearity.". In the OLS model, if both variables are included Twitter follower coefficient turns negative and becomes insignificant. Can I really trust that there is no multicollinearity? $\endgroup$
    – Tapio
    Jan 10, 2023 at 16:47
  • $\begingroup$ I think you are confusing multicolinearity with just "correlation." We expect that independent variables in a regression will be correlated with one another to some extent - that's why we need regression in the first place. Multicolinearity is just what happens when that correlation becomes SO high that it becomes meaningless to try and talk about the effect of one variable "controlling" for the other, because both are functionally measuring the same thing. That's clearly not the case here, either according to the VIF or conceptually. $\endgroup$ Jan 10, 2023 at 19:50
  • $\begingroup$ Thank you again. Would the correct interpretation be that multicollinearity == high VIF? That variance of one variable is excessively explained by another one? The r2 for a model only (tw_likes ~ fb_likes) including those variables is 0.467, which then simply means that there is no multicollinearity issue, end of story. About the rest of the comment, I agree with the sentiment. My goal is to predict and I struggle with making a model which works with "missing data". I guess this is also something I should reconsider? $\endgroup$
    – Tapio
    Jan 10, 2023 at 20:13
  • $\begingroup$ A high VIF (like higher than 5 or 10) is a measure of multicolinearity yes. There isn't any simple relationship between R2 and multicolineairty one way or the other. But I really don't think multicolinearity is what you should be worried about in this problem. Your problem is missing data, and you have to decide conceptually what that missing data means.... $\endgroup$ Jan 11, 2023 at 0:44
  • $\begingroup$ ...sometimes missing data means "zero" (if people are missing values for "how many years of college did you attend" because earlier in the survey they said didn't go to college), and sometimes missing data means "not applicable" ("what color is you cat?" if you don't have a cat") and sometimes it means "value unknown" (if you ask someone their income and they just refuse to answer). Each of those kinds of data needs to be dealt with in a different way. You just need to decide whether to treat "missing twitter likes because wasn't promoted on twitter" as "zero" or "not applicable." $\endgroup$ Jan 11, 2023 at 0:46
1
$\begingroup$

This is not necessarily a complex problem, but unfortunately there are no specific solutions as well. There are many different approaches that you can take which depend on your dataset and your aims.

If you intend to create a composite index to address your problem (which is my preferred method in your problem as well), this can be a valid method but certain assumptions need to be met. First of all, I suggest you check out the "Handbook on Constructing Composite Indicators" by the OECD: https://www.oecd.org/sdd/42495745.pdf . There must be a theoratical basis for how you create your composite index which depends on your planned analysis and your aims.
What I can suggest, is creating a new composite index based on the guidelines provided in the above link. Also, I suggest you create a weight variable and weight your new index based on the number of available social media data (1,2,3 or none).

Check other solutions for addressing high multicollinearity as well. This blog has a good summary: link

Response to edit: When including both variables causes a reversal of signs, which is the case here, multicollinearity is causing significant influence on the coefficients. Depending on what you want to do with your model, this can be an indicator that you need to deal with it.

$\endgroup$
2
  • 1
    $\begingroup$ Thank you for the answer! I will check out the proposed links. I just wanted to notify that I modified the question slightly before I saw your response. I found out that I had calculated the VIF using statsmodels without constant, which resulted in high VIF. After adding the constant the VIF is small. I'm slightly perplexed about the multicollinearity now. Anyways, I will probably find an answer to this also from the links. $\endgroup$
    – Tapio
    Jan 10, 2023 at 13:18
  • $\begingroup$ I updated my answer regarding your edit. $\endgroup$ Jan 10, 2023 at 13:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.