Skip to main content
added 7 characters in body
Source Link
Adrian Keister
  • 6k
  • 8
  • 31
  • 47

Your procedure would need to have some caveats with it, depending on the nature of the causal relationships present.

Case 1: a confounder. Examine this causal diagram:

enter image description here

Here $Z$ sets up a backdoor path $X\leftarrow Z\to Y.$ If you regressed $Y=aX+\varepsilon,$ you would run afoul of the confounder $Z,$ and you would not obtain the correct causal relationship between $X$ and $Y.$ Assuming linear regression is the right way to go, you would need to include $Z$ in your regression: $Y=aX+bZ+\varepsilon.$ This effectively conditions on $Z,$ thus stopping information from flowing through the backdoor path.

Case 2: a mediator. Examine this causal diagram:

enter image description here

Now you have a mediator $M$, and if you were to regress (thinking the situation was the same as the confounder in Case 1) $Y=aX+bM+\varepsilon,$ again you would get the WRONG causal effect of $X$ on $Y.$ Why is that? Because there is no backdoor path from $X$ to $Y$ in this situation, and by conditioning on $M,$ you close off the additional effect that is mediated through $M.$ Incidentally, there are some significant counterfactual methods of analyzing mediators that are worth learning. See, e.g., Causal Inference in Statistics: A Primer, by Pearl, Glymour, and Jewell.

So traditional theories of regression do not talk about causality in this way, nor do they have the causal machinery of backdoor paths to inform when to include variables in the regression, or not.

Perhaps even more basic, is that regression by itself doesn't even tell you which variables are causes, and which are effects! You could regress $Y=mX+\varepsilon,$ thinking of $X$ as the cause and $Y$ as the effect, or you could regress $X=mY+\varepsilon,$ reversing the roles. How do you know which one is right? There's actually a theorem in Causality: Models, Reasoning, and Inference, by Pearl, Theorem 1.2.8, that applies here. It turns out that $X\to Y$ and $Y\to X$ are observationally equivalent: same skeleton, and same $v$-structures (non-existent), which implies the startling conclusion: you cannot statistically distinguish between $X\to Y$ and $Y\to X.$ In other words, data alone cannot help you decide which direction the arrow should be.

Finally, of course, the variables in question might not be related in a regression-style fashion at all. What if neither linear regression of any sort, nor logistic regression of any sort captures the relationship between $X$ and $Y?$ Example: finding the phase anglefrequency $\omega$ in $$Y=a\cos(\omega T+\theta).$$ This is outside linear regression and logistic regression, because the desired coefficients do not show up linearly in the expression. But in this structural equation model, we are clearly thinking of $\theta$ as having a causal effect on $Y.$

So, for these reasons, causal diagrams and causal thinking, while they can certainly apply in regression situations, cannot be reduced to regression situations as a special case.

Your procedure would need to have some caveats with it, depending on the nature of the causal relationships present.

Case 1: a confounder. Examine this causal diagram:

enter image description here

Here $Z$ sets up a backdoor path $X\leftarrow Z\to Y.$ If you regressed $Y=aX+\varepsilon,$ you would run afoul of the confounder $Z,$ and you would not obtain the correct causal relationship between $X$ and $Y.$ Assuming linear regression is the right way to go, you would need to include $Z$ in your regression: $Y=aX+bZ+\varepsilon.$ This effectively conditions on $Z,$ thus stopping information from flowing through the backdoor path.

Case 2: a mediator. Examine this causal diagram:

enter image description here

Now you have a mediator $M$, and if you were to regress (thinking the situation was the same as the confounder in Case 1) $Y=aX+bM+\varepsilon,$ again you would get the WRONG causal effect of $X$ on $Y.$ Why is that? Because there is no backdoor path from $X$ to $Y$ in this situation, and by conditioning on $M,$ you close off the additional effect that is mediated through $M.$ Incidentally, there are some significant counterfactual methods of analyzing mediators that are worth learning. See, e.g., Causal Inference in Statistics: A Primer, by Pearl, Glymour, and Jewell.

So traditional theories of regression do not talk about causality in this way, nor do they have the causal machinery of backdoor paths to inform when to include variables in the regression, or not.

Perhaps even more basic, is that regression by itself doesn't even tell you which variables are causes, and which are effects! You could regress $Y=mX+\varepsilon,$ thinking of $X$ as the cause and $Y$ as the effect, or you could regress $X=mY+\varepsilon,$ reversing the roles. How do you know which one is right? There's actually a theorem in Causality: Models, Reasoning, and Inference, by Pearl, Theorem 1.2.8, that applies here. It turns out that $X\to Y$ and $Y\to X$ are observationally equivalent: same skeleton, and same $v$-structures (non-existent), which implies the startling conclusion: you cannot statistically distinguish between $X\to Y$ and $Y\to X.$ In other words, data alone cannot help you decide which direction the arrow should be.

Finally, of course, the variables in question might not be related in a regression-style fashion at all. What if neither linear regression of any sort, nor logistic regression of any sort captures the relationship between $X$ and $Y?$ Example: finding the phase angle in $$Y=a\cos(\omega T+\theta).$$ This is outside linear regression and logistic regression, because the desired coefficients do not show up linearly in the expression. But in this structural equation model, we are clearly thinking of $\theta$ as having a causal effect on $Y.$

So, for these reasons, causal diagrams and causal thinking, while they can certainly apply in regression situations, cannot be reduced to regression situations as a special case.

Your procedure would need to have some caveats with it, depending on the nature of the causal relationships present.

Case 1: a confounder. Examine this causal diagram:

enter image description here

Here $Z$ sets up a backdoor path $X\leftarrow Z\to Y.$ If you regressed $Y=aX+\varepsilon,$ you would run afoul of the confounder $Z,$ and you would not obtain the correct causal relationship between $X$ and $Y.$ Assuming linear regression is the right way to go, you would need to include $Z$ in your regression: $Y=aX+bZ+\varepsilon.$ This effectively conditions on $Z,$ thus stopping information from flowing through the backdoor path.

Case 2: a mediator. Examine this causal diagram:

enter image description here

Now you have a mediator $M$, and if you were to regress (thinking the situation was the same as the confounder in Case 1) $Y=aX+bM+\varepsilon,$ again you would get the WRONG causal effect of $X$ on $Y.$ Why is that? Because there is no backdoor path from $X$ to $Y$ in this situation, and by conditioning on $M,$ you close off the additional effect that is mediated through $M.$ Incidentally, there are some significant counterfactual methods of analyzing mediators that are worth learning. See, e.g., Causal Inference in Statistics: A Primer, by Pearl, Glymour, and Jewell.

So traditional theories of regression do not talk about causality in this way, nor do they have the causal machinery of backdoor paths to inform when to include variables in the regression, or not.

Perhaps even more basic, is that regression by itself doesn't even tell you which variables are causes, and which are effects! You could regress $Y=mX+\varepsilon,$ thinking of $X$ as the cause and $Y$ as the effect, or you could regress $X=mY+\varepsilon,$ reversing the roles. How do you know which one is right? There's actually a theorem in Causality: Models, Reasoning, and Inference, by Pearl, Theorem 1.2.8, that applies here. It turns out that $X\to Y$ and $Y\to X$ are observationally equivalent: same skeleton, and same $v$-structures (non-existent), which implies the startling conclusion: you cannot statistically distinguish between $X\to Y$ and $Y\to X.$ In other words, data alone cannot help you decide which direction the arrow should be.

Finally, of course, the variables in question might not be related in a regression-style fashion at all. What if neither linear regression of any sort, nor logistic regression of any sort captures the relationship between $X$ and $Y?$ Example: finding the frequency $\omega$ in $$Y=a\cos(\omega T+\theta).$$ This is outside linear regression and logistic regression, because the desired coefficients do not show up linearly in the expression. But in this structural equation model, we are clearly thinking of $\theta$ as having a causal effect on $Y.$

So, for these reasons, causal diagrams and causal thinking, while they can certainly apply in regression situations, cannot be reduced to regression situations as a special case.

Added observational equivalence.
Source Link
Adrian Keister
  • 6k
  • 8
  • 31
  • 47

Your procedure would need to have some caveats with it, depending on the nature of the causal relationships present.

Case 1: a confounder. Examine this causal diagram:

enter image description here

Here $Z$ sets up a backdoor path $X\leftarrow Z\to Y.$ If you regressed $Y=aX+\varepsilon,$ you would run afoul of the confounder $Z,$ and you would not obtain the correct causal relationship between $X$ and $Y.$ Assuming linear regression is the right way to go, you would need to include $Z$ in your regression: $Y=aX+bZ+\varepsilon.$ This effectively conditions on $Z,$ thus stopping information from flowing through the backdoor path.

Case 2: a mediator. Examine this causal diagram:

enter image description here

Now you have a mediator $M$, and if you were to regress (thinking the situation was the same as the confounder in Case 1) $Y=aX+bM+\varepsilon,$ again you would get the WRONG causal effect of $X$ on $Y.$ Why is that? Because there is no backdoor path from $X$ to $Y$ in this situation, and by conditioning on $M,$ you close off the additional effect that is mediated through $M.$ Incidentally, there are some significant counterfactual methods of analyzing mediators that are worth learning. See, e.g., Causal Inference in Statistics: A Primer, by Pearl, Glymour, and Jewell.

So traditional theories of regression do not talk about causality in this way, nor do they have the causal machinery of backdoor paths to inform when to include variables in the regression, or not.

Perhaps even more basic, is that regression by itself doesn't even tell you which variables are causes, and which are effects! You could regress $Y=mX+\varepsilon,$ thinking of $X$ as the cause and $Y$ as the effect, or you could regress $X=mY+\varepsilon,$ reversing the roles. How do you know which one is right? There's actually a theorem in Causality: Models, Reasoning, and Inference, by Pearl, Theorem 1.2.8, that applies here. It turns out that $X\to Y$ and $Y\to X$ are observationally equivalent: same skeleton, and same $v$-structures (non-existent), which implies the startling conclusion: you cannot statistically distinguish between $X\to Y$ and $Y\to X.$ In other words, data alone cannot help you decide which direction the arrow should be.

Finally, of course, the variables in question might not be related in a regression-style fashion at all. What if neither linear regression of any sort, nor logistic regression of any sort captures the relationship between $X$ and $Y?$ Example: finding the phase angle in $$Y=a\cos(\omega T+\theta).$$ This is outside linear regression and logistic regression, because the desired coefficients do not show up linearly in the expression. But in this structural equation model, we are clearly thinking of $\theta$ as having a causal effect on $Y.$

So, for these reasons, causal diagrams and causal thinking, while they can certainly apply in regression situations, cannot be reduced to regression situations as a special case.

Your procedure would need to have some caveats with it, depending on the nature of the causal relationships present.

Case 1: a confounder. Examine this causal diagram:

enter image description here

Here $Z$ sets up a backdoor path $X\leftarrow Z\to Y.$ If you regressed $Y=aX+\varepsilon,$ you would run afoul of the confounder $Z,$ and you would not obtain the correct causal relationship between $X$ and $Y.$ Assuming linear regression is the right way to go, you would need to include $Z$ in your regression: $Y=aX+bZ+\varepsilon.$ This effectively conditions on $Z,$ thus stopping information from flowing through the backdoor path.

Case 2: a mediator. Examine this causal diagram:

enter image description here

Now you have a mediator $M$, and if you were to regress (thinking the situation was the same as the confounder in Case 1) $Y=aX+bM+\varepsilon,$ again you would get the WRONG causal effect of $X$ on $Y.$ Why is that? Because there is no backdoor path from $X$ to $Y$ in this situation, and by conditioning on $M,$ you close off the additional effect that is mediated through $M.$ Incidentally, there are some significant counterfactual methods of analyzing mediators that are worth learning. See, e.g., Causal Inference in Statistics: A Primer, by Pearl, Glymour, and Jewell.

So traditional theories of regression do not talk about causality in this way, nor do they have the causal machinery of backdoor paths to inform when to include variables in the regression, or not.

Finally, of course, the variables in question might not be related in a regression-style fashion at all. What if neither linear regression of any sort, nor logistic regression of any sort captures the relationship between $X$ and $Y?$ Example: finding the phase angle in $$Y=a\cos(\omega T+\theta).$$ This is outside linear regression and logistic regression, because the desired coefficients do not show up linearly in the expression. But in this structural equation model, we are clearly thinking of $\theta$ as having a causal effect on $Y.$

So, for these reasons, causal diagrams and causal thinking, while they can certainly apply in regression situations, cannot be reduced to regression situations as a special case.

Your procedure would need to have some caveats with it, depending on the nature of the causal relationships present.

Case 1: a confounder. Examine this causal diagram:

enter image description here

Here $Z$ sets up a backdoor path $X\leftarrow Z\to Y.$ If you regressed $Y=aX+\varepsilon,$ you would run afoul of the confounder $Z,$ and you would not obtain the correct causal relationship between $X$ and $Y.$ Assuming linear regression is the right way to go, you would need to include $Z$ in your regression: $Y=aX+bZ+\varepsilon.$ This effectively conditions on $Z,$ thus stopping information from flowing through the backdoor path.

Case 2: a mediator. Examine this causal diagram:

enter image description here

Now you have a mediator $M$, and if you were to regress (thinking the situation was the same as the confounder in Case 1) $Y=aX+bM+\varepsilon,$ again you would get the WRONG causal effect of $X$ on $Y.$ Why is that? Because there is no backdoor path from $X$ to $Y$ in this situation, and by conditioning on $M,$ you close off the additional effect that is mediated through $M.$ Incidentally, there are some significant counterfactual methods of analyzing mediators that are worth learning. See, e.g., Causal Inference in Statistics: A Primer, by Pearl, Glymour, and Jewell.

So traditional theories of regression do not talk about causality in this way, nor do they have the causal machinery of backdoor paths to inform when to include variables in the regression, or not.

Perhaps even more basic, is that regression by itself doesn't even tell you which variables are causes, and which are effects! You could regress $Y=mX+\varepsilon,$ thinking of $X$ as the cause and $Y$ as the effect, or you could regress $X=mY+\varepsilon,$ reversing the roles. How do you know which one is right? There's actually a theorem in Causality: Models, Reasoning, and Inference, by Pearl, Theorem 1.2.8, that applies here. It turns out that $X\to Y$ and $Y\to X$ are observationally equivalent: same skeleton, and same $v$-structures (non-existent), which implies the startling conclusion: you cannot statistically distinguish between $X\to Y$ and $Y\to X.$ In other words, data alone cannot help you decide which direction the arrow should be.

Finally, of course, the variables in question might not be related in a regression-style fashion at all. What if neither linear regression of any sort, nor logistic regression of any sort captures the relationship between $X$ and $Y?$ Example: finding the phase angle in $$Y=a\cos(\omega T+\theta).$$ This is outside linear regression and logistic regression, because the desired coefficients do not show up linearly in the expression. But in this structural equation model, we are clearly thinking of $\theta$ as having a causal effect on $Y.$

So, for these reasons, causal diagrams and causal thinking, while they can certainly apply in regression situations, cannot be reduced to regression situations as a special case.

Source Link
Adrian Keister
  • 6k
  • 8
  • 31
  • 47

Your procedure would need to have some caveats with it, depending on the nature of the causal relationships present.

Case 1: a confounder. Examine this causal diagram:

enter image description here

Here $Z$ sets up a backdoor path $X\leftarrow Z\to Y.$ If you regressed $Y=aX+\varepsilon,$ you would run afoul of the confounder $Z,$ and you would not obtain the correct causal relationship between $X$ and $Y.$ Assuming linear regression is the right way to go, you would need to include $Z$ in your regression: $Y=aX+bZ+\varepsilon.$ This effectively conditions on $Z,$ thus stopping information from flowing through the backdoor path.

Case 2: a mediator. Examine this causal diagram:

enter image description here

Now you have a mediator $M$, and if you were to regress (thinking the situation was the same as the confounder in Case 1) $Y=aX+bM+\varepsilon,$ again you would get the WRONG causal effect of $X$ on $Y.$ Why is that? Because there is no backdoor path from $X$ to $Y$ in this situation, and by conditioning on $M,$ you close off the additional effect that is mediated through $M.$ Incidentally, there are some significant counterfactual methods of analyzing mediators that are worth learning. See, e.g., Causal Inference in Statistics: A Primer, by Pearl, Glymour, and Jewell.

So traditional theories of regression do not talk about causality in this way, nor do they have the causal machinery of backdoor paths to inform when to include variables in the regression, or not.

Finally, of course, the variables in question might not be related in a regression-style fashion at all. What if neither linear regression of any sort, nor logistic regression of any sort captures the relationship between $X$ and $Y?$ Example: finding the phase angle in $$Y=a\cos(\omega T+\theta).$$ This is outside linear regression and logistic regression, because the desired coefficients do not show up linearly in the expression. But in this structural equation model, we are clearly thinking of $\theta$ as having a causal effect on $Y.$

So, for these reasons, causal diagrams and causal thinking, while they can certainly apply in regression situations, cannot be reduced to regression situations as a special case.