Let's set up the situation of having some $Y$ that I think depends on a linear combination of $X_1$ and $X_2$. I could fit a regression model:

$$y_i = \beta_0 + \beta_1x_{i1} + \beta_2x_{i2}$$

We could write this as a function of the predictor variables:

$$y(x_1, x_2) = \beta_0 + \beta_1x_{1} + \beta_2x_{2}$$

Then we would interpret the coefficients as being the partial derivatives.

$$\dfrac{\partial y}{\partial x_1} = \beta_1$$

$$\dfrac{\partial y}{\partial x_2} = \beta_2$$

This is consistent with our usual idea that, as we increase $x_1$ by one unit and leave $x_2$ alone, $y$ changes by $\beta_1$.

However, what if $X_1$ and $X_2$ are correlated? In that case, if we increase $x_1$ by one unit, $x_2$ should change by some amount.

I've gotten as far as thinking that it has something to do with the inner product of $\big(\beta_1, \beta_2\big)$ with itself with respect to the covariance matrix of $X_1$ and $X_2$:

$$\begin{pmatrix} \beta_1 & \beta_2 \end{pmatrix} \begin{pmatrix} \sigma_{1,1} & \sigma_{1,2}\\ \sigma_{2,1} & \sigma_{2,2} \end{pmatrix} \begin{pmatrix} \beta_1 \\ \beta_2 \end{pmatrix} $$

Thoughts? How does the intercept play into this? Certainly the intercept should drop out, but where?

(There should be $\widehat{\text{hats}}$ all over the place, yes.)

  • $\begingroup$ Yes, please do add the hats in the $y$ and $\beta$s $\endgroup$
    – Tim Mak
    Commented Apr 8, 2020 at 2:53
  • $\begingroup$ Your characterization of "correlated" sounds more like causally linked. I suppose one way to interpret it would be "if I predicted $X_2$ from $X_1$ using linear regression, and then changed $x_1$ and $x_2$ simultaneously in a way consistent with that prediction, how much would the prediction of $Y$ change?" If that's your intention, what you need is a directional derivative. It gives a nice, simple solution, because it's just the (intuitively obvious) linear combination of the $\hat\beta_i.$ $\endgroup$
    – whuber
    Commented Feb 24, 2022 at 22:57

1 Answer 1


The coefficients in a multiple linear regression are by definition conditional coefficients. If you want the marginal relationship, the general answer is to integrate over the distribution of $x_1$ and $x_2$. However, typically, the distribution is unspecified, and people use the empirical distribution instead. In the linear regression case, I think this reduces to simply fitting the model of one variable without the other.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.