# How to investigate the effect of predictors that correlate using regression?

I have 7 predictors and I would like to know how each of them affects Y variable. However, some of them correlate:

How should I proceed?

1. Fit them individually?

y ~ A

y ~ B

...

Then I can have pseudo effects? E.g. C has an effect in real life only, however, it correlates with D; thus, D shows pseudo effect in regression?

1. Fit them together?

y ~ A + B + C + D + E + F + G

This shows no effects of C and D, since they correlate?

• This depends on the question that you are trying to answer with your analysis. Commented Aug 5, 2020 at 15:25
• The correlation matrix does not indicate type of data you have used. Cor-relatedness should increase the effect on Y dependent variable ?
– user10619
Commented Aug 6, 2020 at 15:15
• If you standardize the data, it could help overcome the effects of cor relatedness on dependent variables !
– user10619
Commented Aug 6, 2020 at 15:20

There are multiple things you can do. I would recommend looking at the problem from the perspective of the new causal revolution. You are interested in the causal effect of $$\{A,B,C,D,E,FF,G\}$$ on $$Y,$$ but you're unsure if there are causal relationships among the explanatory variables. The very first thing I would do is draw a causal diagram. This is just a Directed Acyclic Graph (DAG), where node $$A$$ causing node $$B$$ is represented by the simple arrow $$A\to B.$$ NEVER underestimate the power of a DAG in analyzing cause-and-effect. Once you have your DAG, you can start to think about what would make the most sense to do, to isolate the causal effect in which you're interested. For example: suppose you have the following DAG:

This is called a mediation scenario. You do NOT have a back-door path from $$X$$ through $$Z$$ to $$Y,$$ because the arrow between $$X$$ and $$Z$$ points to $$Z.$$ This is therefore not a confounding situation, even though $$X$$ and $$Z$$ would likely be correlated. There is no need to condition on $$Z.$$ In fact, if you want the true causal effect of $$X$$ on $$Y,$$ you should NOT condition on $$Z.$$ On the other hand, suppose you had this situation:

Now you have a backdoor path: $$X\leftarrow Z\to Y,$$ and you must condition on $$Z.$$

Now I've used this term "conditioning" a couple of times. In a linear regression scenario, conditioning looks like simply including the variable in the model. So in the mediation example (the first one above), not conditioning on $$Z$$ means your model is $$Y=mX+b.$$ In the confounding example (the second one with the backdoor path), conditioning on $$Z$$ means your model is $$Y=mX+nZ+b.$$

This should get you started, I hope. If you draw a DAG for your situation, please do include it in your question.

• Thank you Adrian, your answer is highly useful! Could you please confirm that my understanding is correct. All these predictors (A to G) are showing the offer of different medical services in a certain time. For instance, if Z would be overall health care spending, then this is clearly a backdoor path for each of these offers (more money spent on health care increases the accessibility of these services). Increasing the offer would, of course, require more money, but budget comes always before service purchase. I'll finish on the next comment... Commented Aug 6, 2020 at 7:41
• The only relationship I see between these predictors (A to G), that if the offer of one is high at certain time, then maybe less other offers are needed (they describe somewhat similar medical services, rehabilitation hours, but they are offered in different settings [e.g. outpatient, inpatient, home etc]). Though they can partly cover or replace each other, this is not universal, since different settings are needed since patients have very different needs. Thus, for me it seems that they act as mediators? If offer of all of them is high, thus patients have better chances for rehabilitation. Commented Aug 6, 2020 at 7:48
• You could well be right. Without a DAG, it would be hard for me to say one way or the other. Commented Aug 6, 2020 at 17:09
• Good point! Anybody else needing help with DAGs, I'll recommend these youtube videos (parts 2.1-2.8): youtube.com/playlist?list=PL_onPhFCkVQimvhuSAFrC8VWLEyNygQR5 Commented Aug 7, 2020 at 15:24

The first way is to possibly construct a good theory first. Without a good theory, two things that should not have any good relationship may have a correlation. For example, my electric bill is high may be correlated to that the rice was overcooked, but not the reason for one over another (maybe all of them are due to the third factor, my rice cooker is broken). Having a valid hypothesis with explanation may give you a good groundwork.

That said if you are given a pool of n variables that may or may not correlate, a good way to handle it may be to select criteria, like maximizing the modified R^2, and the first regress the data individually with respect to each of the variable, $$Y$$ ~$$X_i$$ for $$i=1,2,..,n$$. You then select the variable that has the greatest R^2. Without losing generality, assume the variable you choose is $$X_i$$l, then you regress for each $$i$$, with $$Y$$~$$X_1+X_i$$ for $$i=2,3....,n$$. Repeat until you think the model is good enough.

• Thank you so much! The example of rice and electric bill is very good! However, sometimes these causalities are not that clear. Could you please check my comment to Adrian and give your thoughts? Commented Aug 6, 2020 at 7:49
• @st4co4 This is in my opinion why statistics is only an analysis tool but not the only tool. I am no expert in medical professionals and I can offer no educated opinion. Maybe you should ask for guidance from those who are qualified to give an opinion in the causality of the factors. Commented Aug 8, 2020 at 3:51