When can we speak of collinearity

Question

In linear models we need to check if a relationship exists among the explanatory variables. If they correlate too much then there is collinearity (i.e., the variables partly explain each other). I am currently just looking at the pairwise correlation between each of the explanatory variables.

Question 1: What classifies as too much correlation? For example, is a Pearson correlation of 0.5 too much?

Question 2: Can we fully determine whether there is collinearity between two variables based on the correlation coefficient or does it depend on other factors?

Question 3: Does a graphical check of the scatterplot of the two variables add anything to what the correlation coefficient indicates?

Answer 1

1. There is no 'bright line' between not too much collinearity and too much collinearity (except in the trivial sense that $r = 1.0$ is definitely too much). Analysts would not typically think of $r = .50$ as too much collinearity between two variables. A rule of thumb regarding multicollinearity is that you have too much when the VIF is greater than 10 (this is probably because we have 10 fingers, so take such rules of thumb for what they're worth). The implication would be that you have too much collinearity between two variables if $r \ge .95$. You can read more about the VIF and multicollinearity in my answer here: What is the effect of having correlated predictors in a multiple regression model?

2. This depends on what you mean by "fully determine". If the correlation between two variables were $r \ge .95$, then most data analysts would say you had problematic collinearity. However, you can have multiple variables where no two variables have a pairwise correlation that high, and still have problematic collinearity hidden amongst the whole set of variables. This is where other metrics, such as the VIFs and condition numbers come in handy. You can read more on this topic at my question here: Is there a reason to prefer a specific measure of multicollinearity?

3. It is always smart to look at your data, and not simply numerical summaries / test results. The canonical reference here is Anscomb's quartet.

Answer 2

My take on the three questions is

Question 1 What classifies as too much correlation? For example: a pearson correlation of 0.5 is that too much?

Many authors argue that (multi-)collinearity is not a problem. Take a look here and here for a rather acid opinion on the subject. The bottom line is that multicollinearity does not have an impact on the hypothesis testing other than having a lower (effective) sample size. It will be hard for you to interpret the regression coefficients if you do a regression, for instance, but you do not violate any basic assumption if you choose to do so.

Question 2 Can we fully determine whether there is collinearity between two variables based on the correlation coefficient or does it depend on other factors?

I think there are several ways of measuring the correlation between two variables, from calculating Pearson's correlation coefficient (if you assume linearity, and apparently you did so), to Spearman's rank, distance correlation, and even doing PCA on your dataset. But I would leave the answer of this question to better informed people than me.

Question 3 Does a graphical check of the scatter plot of the two variables add anything to what the correlation coefficient indicates?

IMO, the answer is sound no.

Answer 3

A common way to evaluate collinearity is with variance inflation factors (VIFs). This can be achieved in R using the 'vif' function within the 'car' package. This has an advantage over looking at only the correlations between two variables, as it simultaneously evaluates the correlation between one variable and the rest of the variables in the model. It then gives you a single score for each predictor in the model.

As stated above there is no hard and fast cutoff, but VIF scores are often decided to be problematic once they are between 5-10. I use field specific rules of thumb for this. Also- there is nothing necessarily invalid about using correlated predictors (so long as they are not perfectly correlated). You will just need more data to separate effects. When you don't have enough data there will be large uncertainties in the parameter estimates of the correlated predictors, and these estimates will be sensitive to re-sampling.

To answer your questions specifically:

1. Don't use correlation coefficients. use VIFs of the model with all predictors and no interactions. VIFs of 5-10 are indicating too much correlation, your specific cutoff depends on what you need to do with the model.

2. It depends on the other predictors in the model, which is why it is beneficial to use VIFs.

3. Nope! The statistics will better quantify what you are eyeballing with the scatter plot. Unless there is a super violation of the assumptions of OLS when regressing your predictors against each other.