1
$\begingroup$

Suppose I'm trying to figure out what makes people shop at a particular store. I conduct a survey where they are asked:

  1. how often they shop there
  2. to pick 3 out of a list of 12 characteristics that they most associate with the store, e.g. "well-known brand", "good quality", "good loyalty programs", etc.

I was thinking of doing a linear regression of Q1 on Q2. in order to figure out which of these factors are significant when making a decision. Denote by $Y$ the frequency and by $X_i$ a binary variable indicating whether the respondent associated factor $i$ with the store:

$$Y \sim 1 + X_1 + ... + X_{12}$$

The problem with this seems to be perfect multicollinearity: $\sum_i X_i = 3$. Curiously, when I ran the regression in R, I checked the model with car::vif and didn't get alarmingly large values (mostly around 2 for each coefficient).

Apart from that, it's difficult to interpret the results... the values of the coefficients don't really correspond to anything, they only give a rough idea of the factors' relative importance. For example, if the average value of $Y$ is 4.5, I get an intercept of 4.0 and the coefficients all approximately 0.1-0.2, so that picking three of them would roughly correspond to the average. In a sense, the regression compares the respondents to a hypothetical person who "didn't select anything", even though that's impossible.

  • Is this model even appropriate at all, given the collinearity problem? If not, what might be a good alternative?
  • I was wondering if forcibly removing the intercept from the model might be a good idea here to try and salvage it?
Improve this question
$\endgroup$
5
  • $\begingroup$ You didn't need an intercept in the first place, because it's automatically included in the $X_i,$ as you observed. Thus, "forcibly removing" it doesn't change the model--it only causes R to use a different model matrix. If indeed the variables are collinear (that is, everybody has provided three answers), then R has already dropped one of your variables already--most likely $X_{12}.$ $\endgroup$
    – whuber ♦
    Commented Apr 5, 2018 at 14:47
  • $\begingroup$ @whuber So do I have multicollinearity here or not? The answer says I don't, but I can't see how the definition isn't satisfied (for all observations $-3 + \sum X_i = 0$)... on the other hand R didn't remove any variables (I checked), and the vif doesn't indicate multicollinearity.. what gives? $\endgroup$
    – Spine Feast
    Commented Apr 6, 2018 at 8:51
  • $\begingroup$ If R removed no variables, then the situation is not as you described it! $\endgroup$
    – whuber ♦
    Commented Apr 6, 2018 at 13:24
  • $\begingroup$ @whuber I suppose there might've been some people who failed to actually select three options. You're saying that if this weren't the case, R would detect it? $\endgroup$
    – Spine Feast
    Commented Apr 6, 2018 at 14:43
  • $\begingroup$ You can easily check: tabulate the values of $\sum X_i.$ If you only get the value $3$, then you have collinearity. $\endgroup$
    – whuber ♦
    Commented Apr 6, 2018 at 14:54

1 Answer 1

Reset to default
1
$\begingroup$

First, I am uncertain you have a multicollinearity problem, and you definitely do not have perfect multicollinearity. This would suggest that knowing one value, you know all the others. If I selected the 1st choice, I have know idea which of the remaining 11 choices were and were not select. Likewise, if I select two specific choices, I still have no idea of which of the remaining 10 was chosen as the 3rd and final option. That said, the proposed model is not unreasonable (though a closer analysis of your research questions and analysis plan would be warranted).

Second, depending on how Y was answered (ordered categories, counts, etc.), you may want to consider a more general model than just a linear regression (e.g., an ordinal logistic model or a count model like a Poisson).

Third, the intercept does have a meaningful interpretation (as I understand your context). Though you forced respondents to select three options, you could have run the analysis where respondents would have been allowed to select from 0 to 12 options. Thus, the intercept should indicate the frequency when none of the options are considered important.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Yes, but that would be a different model then. When they are forced to choose 3 associations, I'm not convinced about that interpretation... what I tried doing is to force the intercept to be equal to Y's mean in the model. This allows the coefficients to be interpreted in the following way: the sum of the 3 chosen coefficients gives the deviation from the mean frequency... is that right? Also, I guess I'm confused about multicollinearity - I thought that occurred when there was an exact linear dependence between the regression variables..? $\endgroup$
    – Spine Feast
    Commented Apr 5, 2018 at 14:19
  • 1
    $\begingroup$ I would agree with your interpretation...though I would say the sum of the 3 chosen coefficients gives the average deviation for such respondents from the mean frequency. As for multicollinearity, you are correct...but you don't have that here. For example, there is no way to have $X1 = c_2·X_2 + c_3·X_3 + ··· + c_{12}X_{12}$ with the data as you've described it. $\endgroup$
    – Gregg H
    Commented Apr 5, 2018 at 15:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.