2
$\begingroup$

Consider the following DAG which shows the direct and indirect effect of $U$ on $Y$.

The total effect of $U \rightarrow Y$ is simply $(2\times4) + 3 = 11$.

I am looking for the derivation of the biased effect of $A \rightarrow Y$.

When I run simulation I get a biased effect of about 5.5

How can I retrieve mathematically this 5.5?

enter image description here

# R simulation

n = 100000
u = rnorm(n, 0, 1)
a = u*2

y = 3*u + 4*a

lm(y ~ a) # biased effect
lm(y ~ u) # total effect
lm(y ~ a + u) # correct effects

2 ways to compute the bias effect, which is what I observe from the simulation, I worked out but that do not make sense to me why it works. Is there a general formula to compute expected bias?

  1. Divide the total effect of $U$ on $Y$, by the path $U \rightarrow A$, $\frac{11}{2}=5.5$

  2. Let's label the paths: $U \rightarrow A = a$, $A \rightarrow Y = b$, and $U \rightarrow Y = c$, then we can compute the bias with: a + ($\frac{c}{b}$), $4 + \frac{3}{2} = 5.5$

$\endgroup$
7
  • 1
    $\begingroup$ Hmm, that's interesting. I would have expected the biased effect to be simply $7=3+4.$ It's certainly true that the total effect of $U$ on $Y$ is correct, and that if you want the unbiased effect of $A$ on $Y$, you must include $U$ in the regression to correct for the confounder. $\endgroup$ Dec 28, 2021 at 22:07
  • 1
    $\begingroup$ strange no? I had the same intuition as you but then I started working out example and it didn't worked! Look at my two solutions and see if you can make sense why they work. $\endgroup$
    – giac
    Dec 28, 2021 at 22:33
  • $\begingroup$ Well, I'm not sure I can follow the theoretical reason why your solutions produce $5.5.$ Why divide by $a=2$ anywhere? +1, by the way, simply for including a DAG. $\endgroup$ Dec 29, 2021 at 15:50
  • 1
    $\begingroup$ @AdrianKeister, the result 5.5 is the result from the simulation. I just can't find a general formula about how to compute expected bias in a simple setting like this. $\endgroup$
    – giac
    Dec 30, 2021 at 14:58
  • 1
    $\begingroup$ Right. I'm afraid I don't have any explanations, either. I agree that $5.5$ appears to be the simulation values (I copied your code and ran it myself and got the same results you did.) But I don't have a theoretical explanation of why $5.5$ should be the result. $\endgroup$ Dec 30, 2021 at 23:11

2 Answers 2

2
+50
$\begingroup$

The OLS coefficient is the ratio of covariance to variance. That is, let $$ y = \beta a + \nu. $$

Then $\beta = \frac{Cov (y,a)}{Var(a)} = \frac{Cov(11u,2u)}{Var(2u)} = \frac{22}{4} \frac{Var(u)}{Var(u)} = 5.5$.

(We can skip the intercept as both variables have mean 0)

More details for example here: https://jrnold.github.io/intro-methods-notes/regression-anatomy.html

$\endgroup$
2
  • $\begingroup$ Nice one too thanks. Could you have a look at my solution too? I was just more thinking of it in mediation term. Thanks $\endgroup$
    – giac
    Jan 2, 2022 at 17:11
  • 1
    $\begingroup$ @giac Keeping things in line with your notation, we have in general $a = \alpha \cdot u$ and $y = \delta u + \beta a$. So the covariance formula from above implies $ \frac{Cov(\delta u + \beta \cdot \alpha u,\alpha u)}{Var(\alpha u)} = \frac{ (\delta + \beta \alpha)\alpha}{\alpha^2} = \frac{\delta}{\alpha} + \beta$, which is the same as your answer. So your approach is correct, I think. $\endgroup$
    – Banach
    Jan 2, 2022 at 17:23
2
$\begingroup$

We start with this

\begin{equation} \begin{split} U &= 1 \\ X &= 2 U \\ Y &= 3 U + 4 X \\ \end{split} \end{equation}

Re-arranging the equation we can derive the bias like this (we can call $X'$ the biased effect of $X$)

\begin{equation} \begin{split} Y &= 3 U + 4 X \\ Y - 4 X &= 3 U \\ Y - 4 \underbrace{X}_{2U} &= 3 U \\ Y - 4 \times 2 U &= 3 U \\ Y - 4 &= \frac{3 U}{2 U} \\ Y &= 4 + 1.5 \\ Y &= 5.5 X' \\ \end{split} \end{equation}

We could derive a formula with a general DAG like this

enter image description here

\begin{equation} Y = \beta + \frac{\delta}{\alpha} \end{equation}

We can theoretically predict the size of a spurious association in a DAG like this with this formula

enter image description here

$\beta=0$

\begin{equation} Y = \frac{4}{2} X' \end{equation}

It works!

n = 1000
u = rnorm(n, 0, 1)
x = 2*u + rnorm(n, 0, 0.01)
y = 4*u + rnorm(n, 0, 0.01)

lm(y~x)
# Spurious association of 2!
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.