Correlation versus cause-effect regression

Question

I know correlation does not imply causation. I have read it nth time. (i.e. weight does not cause height etc. etc.)

However, to find the effect of a moderator variable on X-Y relationship, a regression model is used such as the GLM in SPSS to test for interaction or multiple regression. See here.

My Question: If the X-Y relationship is a correlation, then why is a cause-effect model used in this instance?

As far as I understand, it makes little sense to classify a variable as independent or dependent in a correlation analysis.

I apologise if this seems like a 'silly' question. In my past life, I had often told my students that there are no silly questions; just questions!

Answer 1

The calculations underlying the correlation and regression are not the same. They allow us to study the relation between two variables in different and complementary ways.

I don't think that it makes sense to say "the X-Y relationship is a correlation" (no more than to say that the relationship is a regression). The correlation is not a characteristic of the nature of the relationship. It is a just coefficient that you decide to use in order to quantity... what ?

• The correlation coefficient is a measure of association between two variables, considering symmetrical roles. We just want to know if there is an association between these two variables and quantify the intensity of the relation. Thus, considering variables X and Y, r(X,Y) = r(Y,X). In this case, it doesn't make sense to distinguish independent variable and dependent variable.

• Regression analysis allows us to study the association between two variables, by studying the variations of one based on the values ​​of the other, i.e., variations of a dependent variable according to the values of an independent variable. In this case, Y=aX+e ≠ X=aY+e (where a is a constant and e an error term). The regression of Y on X is different from the regression of X on Y. In this case, we have to specify a dependent variable and an independant variable, according to our hypotheses.

Correlation is often a first step of descriptive analysis (if there is no correlation, it is not useful to go ahead in regression analysis). In regression, we can test an hypothesis concerning the relation between a dependent variable and a independent variable (and also moderation effects of a third variable).

In addition, regarding the test of causal relations, the problem lies more in the nature of the variable (you can study the effect of gender on school achievement, but not vice versa) and the study design (causality implies that a variable temporally precedes another).

Answer 2

Causality is more a matter of interpretation than a matter of using a given method. I will try to illustrate using the example you mention.

If you are investigating the link between height and weight, you can start by computing Pearson's correlation coefficient. As an alternative you could have used linear regression to estimate the following equation: $height = a + b*weight$. This model gives you the same information than Pearson's correlation coefficient. Indeed, the R-squared of this model is the square of Pearson's correlation coefficient. Moreover, if the variables $height$ and $weight$ are standardized, the coefficient $b$ corresponds to the correlation coefficient computed previously.

All these methods yield the same result. All the methods are correct. You would have come up with the same conclusion if you had estimated the model $weight = a + b*height$.

However, it is problematic and even wrong when you start saying that if a 1 kilogram increase in weight, will increase height by $b$ centimeters. Then you give a causal interpretation to the model.

Answer 3

No statistical method can show causation. There are study designs that are more or less susceptible to various problems in the attempt to discern causation.

In particular, we can distinguish observational studies from experiments.

Observational studies have the problem that there can always be some other variable that is moderating or mediating the relationship between Y and X. There are many examples of this. Here is one:

Heavier children have larger vocabularies than lighter children.

Let Y be "number of words in vocabulary" and X be "weight". Then a regression can show the relationship. Weight is the independent variable and vocabulary is the dependent variable. Unfortunately, we have forgotten about another variable: Age. Age mediates the relationship between vocabulary and weight, and if we add age as another independent variable (say $X_2$ where weight is $X_1$) then we get

$Y = a + b_1*X_1 + b_2*X_2$

here, I would wager that $b_1$ would be close to 0.

One key aspect of experiments is that the researcher assigns the treatment randomly. The problem is that many questions can't really be studied in this way: How could we randomly assign children to different weights?