When should we concern the correlations among variables? [closed]

Question

In some papers, I saw researchers perform the correlation among variables table (sometimes also called a correlation matrix), but in some papers I did not see the same thing. So, when should we perform correlation test among independent variable?

Answer 1

I think this question will become clear to you if you think about what correlation is and what it is not. To do that I think it's important we talk about some key differences in ideas. The first difference I want to highlight is between correlation and causation. In many research papers, we are not really interested in knowing whether two variables are correlated. For example, we know that education level and wage are correlated, but what we really want to know is does a higher education level cause a person to have a higher wage. Now this difference is typically more applicable when we talk about correlations and accusations between independent and dependent variables. This topic has been covered rather extensively on this website so I will link some useful answers at the bottom.

For independent variables, we may not be concerned whether or not our covariates cause one and another but we are interested in having some other nice properties between the variables. For one, notice that we call these variables independent variables. The first thing you should realize is that in the population if our variables are truly independent then we have that,

$$\mathbb{E}[X_1X_2]=\mathbb{E}[X_1]\mathbb{E}[X_2]$$

This will imply that the covariance is zero,

$$Cov(X_1,X_2)=\mathbb{E}[X_1X_2]-\mathbb{E}[X_1]\mathbb{E}[X_2]=\mathbb{E}[X_1]\mathbb{E}[X_2]-\mathbb{E}[X_1]\mathbb{E}[X_2]=0$$

And so the correlation will be,

$$\rho(X,Y)=\frac{Cov(X_1,X_2)}{\sqrt{Var(X_1)Var(X_2)}}=0$$

However, this is in the population. When computing a correlation table (or matrix) we are really computing the Pearson or Spearman correlation. The Pearson correlation is very similar to the above formula for correlation except that we replace the population moments with their sample analogues (i.e. expectations become means). That means that even though we may (and most likely in practice will) have non-zero values for our correlations between variables. However, these correlations may not be all that interesting to the statistician, especially if we truly believe that in population these variables are independent. Furthermore, even if we got a zero correlation in the sample that does not serve as proof that the variables are indeed independent!

Finally, let me conclude by saying sometimes correlations are useful in research! For example, an important assumption in the instrumental variables model (a popular method for determining causal effects) is that of relevance which implies $Cov(Instrument, Endogenous Variable)\neq 0$. And sometimes we want to know how closely correlated our available independent variables are. Here are some examples of this.

To conclude, the reason not all researchers will display these graphs is that it is not always useful for the particular goal of the paper. An example of this might be when a researcher is interested in causation. Still, there are many cases when these tables might prove useful but it depends on the goal of the researcher.

Some useful previous answers:

Causation vs. Correlation

Independence vs. Correlation

Some other useful links:

Causation vs. Correlation Details