Let's say I know the relation between x and y is linear yet noisy. Given a noisy (x,y) dataset is there a way to deduce if it was most likely generated out of an underlying function

(a) $y = a + b \times x + noise$ or via (b) $x=c + d \times y + noise$

I'm not sure if I know the nature of the noise but let's assume it is gaussian and non correlated.

Normally I would use regression to determine the coeficents and I do know that the estimated parameters differ depending on whether x is taken to be the independent variable or y.

But I don't know if there is a statistical way to differentiate the two cases (a) and (b) that I described.


1 Answer 1


Not as far as I know. Which variable is dependent and which is independent depends on what the variables are. It can be determined (if at all) by arguments outside of statistics.

Sometimes, it's just that one "makes sense" and the other doesn't. If we have data on height and weight of human adults, it "makes sense" for weight to depend on height, but not the other way around.

Sometimes the temporal order of the variables determines which is which: Something that happens earlier cannot depend on something that happens later. For example, if the variables are "who you voted for in 2012" and sex, only one relationship makes any sense at all (I voted for Obama and therefore I'm female????)

Sometimes there is mutual causation. Does depression cause anxiety? Does anxiety cause depression? Probably both.

And sometimes there is no way to tell at all. Indeed, sometimes, there is no dependency in the relationship because, e.g. both variables depend on something else.

  • $\begingroup$ Thanks. I, of course, agree with all what you wrote. Yet I was wondering if one might be able to exploit the nature of noise in the output to discriminate? Your answer makes it seem probably not. $\endgroup$ Commented Apr 14, 2013 at 12:29
  • $\begingroup$ Also, though causation is a trait of the physical process ultimately, yet there are statistical techniques to hint as to the direction of causation from a given dataset, right? $\endgroup$ Commented Apr 14, 2013 at 12:34
  • $\begingroup$ What "statistical techniques to hint at the direction of causation" did you have in mind? In part, it depends on how broad your definition of "statistical technique" is. Randomization certainly helps, but I think of that more as research design. $\endgroup$
    – Peter Flom
    Commented Apr 14, 2013 at 12:41
  • $\begingroup$ Grainger causality for instance....en.wikipedia.org/wiki/Granger_causality $\endgroup$ Commented Apr 14, 2013 at 13:16
  • $\begingroup$ @curious: Path analysis might be interesting to you if you haven't already come across it, but it can't determine the direction of causality between a single pair of variables. Hill's criteria are a very useful guide - but more for assessing a research programme than a single sample. $\endgroup$ Commented Apr 15, 2013 at 11:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.