0
$\begingroup$

I struggling with some analysis I'm doing and would really appreciate some insights:

I'm measuring people's the dependency of "Environmental attitude" (DV) on number of hours of sports activities they are doing "out in the nature" (IV 1) and inside a building (IV 2) [the data is from a survey N~500). When I run a linear regression with these two I get that IV1 (nature hours) has positive significant effect on the DV, and IV2 does not.

Then I added another IV: the ratio between time the activities in the following form:

IV3 = ((time outside)-(time inside))/((time outside)+(time inside)) + 1

so it is a variable the ranges from 0 to 2, where 0 indicates only inside activities and 2 indicates only outside activities (and 1 is equal amount of time).

Obviously IV3 is highly correlated to IV1 and IV2. When I run a regression with the 3 of them, the R-square gets better (more of the variance is explained) but to my surprise IV2 (hours inside, which wasn't significant before) becomes significant as well as IV3 (the ratio) while IV1 becomes not-significant.

Why do you think that is happening? How would you approach this analysis?

Thanks!

$\endgroup$
  • $\begingroup$ Welcome to CV. Since you’re new here, you may want to take our tour, which has information for new users. I am not sure why you think that your analysis has anything to do with interaction and mediation as your title indicates. Are you asking about how to include an interaction term or how to conduct a mediation analysis? Could you provide more information in that respect? $\endgroup$ – T.E.G. Jun 19 '17 at 4:51
  • $\begingroup$ Yes, thanks for the comment- My doubts are about the interpretation of the results - since IV3 is highly correlated with IV1,2 it seems very "biased" to take the full model results as is. How should I account for possible mediation/multi-linearity? (in the full model IV3, the ratio, get a Tolerance < 0.4) my intuition says I must somehow take that into account because IV3 is literally composed of IV1&IV2 (not linearly, but still). Thanks! $\endgroup$ – Shay Jun 19 '17 at 5:06
0
$\begingroup$
  1. Don't ever be surprised that $r^2$ increases when you add a new variable to the regression, even one that's highly correlated with other predictors. If the new variable is a linear function of the old independent variables (i.e. if you had just included IV1 - IV2 as a new variable), then your software would probably give you an error message. Otherwise, the new independent variable will "soak up" some variance and increase $r^2$. If you haven't seen this before, generate some completely random numbers, add them as a new independent variable to your regression, and watch $r^2$ increase!

  2. Hard to say exactly why the "significance" changes when you add IV3 to the model. One possibility is that you can almost compute IV1 as a linear combination of IV2 and IV3, and that approximation of IV1 does a better job predicting than IV1. But this isn't really what you should be focusing on. You need to ask yourself, "What is my scientific hypothesis, and how should I go about fitting a model that can test this hypothesis?"

For instance, if you simply think that the DV is positively associated with IV1 and negatively with IV2, just include those 2 in the model. Your initial result doesn't surprise me: people who exercise outdoors more probably do love the environment more, and people who exercise indoors more probably don't dislike the environment (they're still active), they just like to lift or play basketball in the gym or what have you. If you suspect there is something more complicated at play, you might fit an "interaction term."

Funky things happen when you start adding highly correlated variables to models. Rather than testing a bunch of models and seeing what's significant and what isn't (this is bad scientific practice), build a model that stems from the scientific background and diagnose it carefully before trying something else.

$\endgroup$
  • $\begingroup$ There are theoretical reasons to believe that the ratio between inside and outside activity (IV3) should be relevant as well in predicting the DV. However, $\endgroup$ – Shay Jun 19 '17 at 4:56
  • $\begingroup$ There are theoretical reasons to believe that the ratio between inside and outside activity (IV3) should be relevant as well in predicting the DV. My doubts are about the interpretation of the results - since IV3 is highly correlated with IV1,2 it seems very "biased" to take the full model results as is. How should I account for possible mediation/multi-linearity? (in the full model IV3, the ratio, get a Tolerance < 0.4) Thanks! $\endgroup$ – Shay Jun 19 '17 at 5:03
  • $\begingroup$ The coefficients are technically "unbiased" assuming that the model is correct (which it never really is :P ). But right, you can't interpret each IV independently: it makes no sense to say "holding IV1 and IV2 constant, a unit change in IV3 is associated with a change in DV of..." Audience depending, you could maybe say, "When IV1 is equal to $x_1$, a unit change in IV2 is associated with a change of $y$ in DV (incorporating both the IV2 and IV3 coefficient), and when IV1 is equal to $x_2$, the relationship changes to __ because __ (IV3 is different now that IV1 has changed)." Does that help? $\endgroup$ – Sheridan Grant Jun 19 '17 at 5:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.