1
$\begingroup$

I have a regression where I find the effect of x on y while controlling for z:

y = x + z

Theoretically,

  1. Both x and z have an effect on y.
  2. z has an effect on x.
  3. But x do not have an effect on z.

Is the simple OLS regression as shown in the above form is valid to capture pure effect of x on y?

Improve this question
$\endgroup$
9
  • 1
    $\begingroup$ N.B: if this question is a duplicate, you should have found it via the search function ;-) $\endgroup$
    – Andy
    Oct 20, 2016 at 13:16
  • $\begingroup$ stats.stackexchange.com/questions/38093/… I found this but it does not address my issue of theoretical causality. $\endgroup$
    – Mumbo.Jumbo
    Oct 20, 2016 at 13:17
  • 1
    $\begingroup$ A risk you might have is that if x and z are correlated (and you say they are in point 2that they both have very similar contributions to y. OLS will give very unstable results if that is the case. You can read more about this: Multicollunearity $\endgroup$
    – JAD
    Oct 20, 2016 at 13:23
  • $\begingroup$ @JarkoDubbeldam yes, that is also a problem, apart from my aim of estimating the pure effect. $\endgroup$
    – Mumbo.Jumbo
    Oct 20, 2016 at 13:27
  • $\begingroup$ And about the correlation for different measurements (I assume you meant that instead of estimations) at 10 different times, you should account for this using mixed models. $\endgroup$
    – JAD
    Oct 20, 2016 at 13:29

1 Answer 1

Reset to default
3
$\begingroup$

Like I said in the comments, the correlation between $x$ and $z$ may cause some issues, but that doesn't mean they are necessarily a problem for your model.

Assuming they are not perfectly correlated (in which case calculating OLS is impossible) the model as is can still be used, depending on what you want with it. If you want to make predictions on $y$, in most cases you can still do that. If you want to draw conclusions about the effect of $x$ on $y$, then you should be more careful.

Is the correlation a problem?

You can check this by checking the VIF (Variance Inflation Factor) of the variable. Most statistical software will have built-in functions to let you calculate this. If the values of the correlated variables $x$ and $z$ are higher than 5, then the correlation is starting to become a problem.

How to fix it

At the cost of introducing a bit of bias, you could drop $z$ from the model. If it is correlated enough with $x$ to become a problem for parameter-estimation, you can assume that their effects on $y$ are pretty similar, so you can just attribute all the predictive power to $x$. Alternatively you can create a new variable $w = x+z$, and use that for your regression.

More information about multicollinearity can be found here and here.

Improve this answer
$\endgroup$
3
  • $\begingroup$ How about regressing x=z and taking residuals and then regressing y=residual? $\endgroup$
    – Mumbo.Jumbo
    Oct 20, 2016 at 14:15
  • 1
    $\begingroup$ That would remove all the information shared by both $x$ and $z$ from the model as you regress only on the part of $x$ that is not explained by $z$. Since you are mostly interested in the effect of $x$ (correct me if I'm wrong), you could instead regress $z\sim x$ and then $y \sim x + res$. I am not 100% sure, but I feel like that should work. (maybe try testing the validity using Cross-validation) $\endgroup$
    – JAD
    Oct 20, 2016 at 14:19
  • $\begingroup$ I have realized after your suggestion that perhaps I can use the mixed model but as you can see that my Data has a multi-level response as well as a multi-level explanatory variable with levels being same, It would be nice to know your suggestions. stats.stackexchange.com/questions/242784/… $\endgroup$
    – Mumbo.Jumbo
    Oct 28, 2016 at 13:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.